Author SHA1 Message Date
ElebobaandClaude Opus 5.5 55286495cb feat(status): раздел «Статус» — контейнеры, карточка, проверки (#14)
CI / ruff · mypy · pytest (pull_request) Successful in 1m11s
- docker-socket-proxy в Compose (только GET /containers/*), сокет только у него
- app/runtime/docker.py: read-only клиент; видны только контейнеры с меткой
  tgit.managed=true, чужие id отклоняются до запроса
- app/runtime/checks.py: running, health, рестарты за окно, CPU/RAM, ошибки
  в логе, HTTP-проба; у каждой проверки свой таймаут, отчёт всегда полный
- app/runtime/snapshots.py: снапшоты раз в 15 с в container_snapshots
- бот: список с 🟢🟡🔴⚪, карточка, «🩺 Запустить проверки», «🔄 Обновить»
- деплой ставит метки tgit.sha / tgit.deployed_at на контейнер бота
- имя джоба CD без неотрисованного выражения

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 11:29:23 +03:00
Eleboba 22d9f0c5fe Merge pull request 'ci: CI бота на Gitea Actions (как в Vobot)' (#42) from ci/41-gitea-actions into dev
CD / deploy → ${{ github.ref_name == 'main' && 'prod' || 'dev' }} (push) Successful in 20s
CI / ruff · mypy · pytest (push) Successful in 1m1s
Reviewed-on: #42
2026-10-17 10:51:48 +03:00
Eleboba da01066d5d Merge branch 'dev' into ci/41-gitea-actions
CI / ruff · mypy · pytest (pull_request) Successful in 1m24s
2026-10-17 10:49:52 +03:00
Eleboba 29f9cbe538 Merge pull request 'feat(infra): CD бота на сервер Vobot через Gitea Actions' (#43) from ci/12-cd into dev
CD / deploy → ${{ github.ref_name == 'main' && 'prod' || 'dev' }} (push) Successful in 54s
Reviewed-on: #43
2026-10-17 10:49:44 +03:00
ElebobaandClaude Opus 5.5 164f055c5e docs: деплой (DEPLOY.md), ADR-015 — CI/CD бота на Gitea Actions
Refs #12

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 00:23:20 +03:00
ElebobaandClaude Opus 5.5 526eb56c82 feat(bot): Telegram через прокси (TELEGRAM_PROXY_URL); лимит памяти бота 384m
С сервера Vobot api.telegram.org недоступен (ни IPv4, ни IPv6) — как и meditron, ходим
через прокси; aiohttp-socks для http/socks. Бот в простое ~190 МБ — 256m мало.

Refs #12

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 00:21:36 +03:00
ElebobaandClaude Opus 5.5 df5224e813 feat(infra): CD бота на сервер Vobot — Gitea Actions + deploy.sh
- infra/deploy/deploy.sh <dev|prod> [ref]: git archive → scp → remote.sh на сервере;
  руками и из CI, хост — DEPLOY_HOST (любой Docker-хост); каждый шаг с ретраями
- remote.sh: атомарная замена app/ (app.old для отката), compose -p tgit-<env> up --build,
  ожидание healthy, REVISION, подсказка отката при неудаче; безопасно перезапускать
- окружение без /srv/tgit/<env>/.env пропускается (prod — когда будет свой токен)
- .gitea/workflows/deploy.yml: push в dev/main → deploy; guard «актуален ли пуш», ключ через env

Refs #12

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 00:18:51 +03:00
ElebobaandClaude Opus 5.5 12b2ca22a5 ci: CI бота на Gitea Actions — ruff, mypy, миграции на Postgres, pytest
CI / ruff · mypy · pytest (pull_request) Successful in 1m28s
Как в Vobot: PR и push в dev/main → джоб «ruff · mypy · pytest»; uv через pip (setup-uv на
self-hosted Gitea не работает); миграции up/down/up на сервисе postgres:16 без маппинга портов.

Refs #41

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 00:13:47 +03:00
Eleboba d1addd2c25 Merge pull request 'feat(bot): кнопка «🚀 Закоммитить» прямо в папке' (#40) from feat/commit-button into dev
Reviewed-on: #40
2026-10-16 00:07:44 +03:00
ElebobaandClaude Opus 5.5 9e1d6322b6 feat(bot): кнопка «🚀 Закоммитить» прямо в папке
При непустом черновике в папке — «🚀 Закоммитить · N»: нет сообщения → спрашивает его и
сразу показывает подтверждение пуша; есть → сразу подтверждение. На подтверждении —
«✏️ Изменить сообщение». Раньше Push был спрятан в «🧾 Черновик» → «💬» → «🚀».

Refs #9

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 00:06:52 +03:00
Eleboba 595368b0a2 Merge pull request 'chore: локальный запуск (make run, make demo-repo) + fix относительного GIT_CLONE_DIR' (#39) from chore/local-run into dev
Reviewed-on: #39
2026-10-15 23:56:53 +03:00
ElebobaandClaude Opus 5.5 0f27f405a7 chore: локальный запуск без Docker и SourceCraft; fix: относительный GIT_CLONE_DIR
- make demo-repo: локальный bare-репозиторий загрузок из infra/uploads-template
- make run: бот на этой машине (SQLite, настройки из .env)
- fix: при относительном clone_dir клон уезжал в data/data/uploads (git clone
  запускается из родительской папки) — путь теперь абсолютный; тест на это
- data/ в .gitignore

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 19:53:40 +03:00
Eleboba b5fbbaf904 Merge pull request 'feat(bot): Коммит — сообщение, Push, конфликты' (#38) from feat/9-push into dev
Reviewed-on: #38
2026-10-15 19:30:23 +03:00
ElebobaandClaude Opus 5.5 ae0cd5d42d feat(bot): Коммит — сообщение, Push, конфликты, публикация в репозиторий загрузок
- Publisher: пуши по одному (lock), fetch → проверка конфликта с base_sha → чистое дерево
  origin/main → черновик → коммит от tgit-bot с трейлером Telegram-User → push;
  отказ push → rebase и повтор (до 3 раз); защита от путей вне клона; лимит 20 пушей/час
- «💬 Сообщение коммита» (1–200 символов, первая строка ≤ 72) и «🚀 Push» с подтверждением;
  исходы: запушено (sha, ссылка, ⏳ CI) · конфликт → «Перезаписать» · нечего коммитить · лимит · сбой
- запись в pushes (chat_id, message_id — для обновления статуса CI/CD в #11), черновик очищается
- ruff: RUF001–003 выключены — кириллица в текстах это норма

Refs #9

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 19:23:09 +03:00
ElebobaandClaude Opus 5.5 614f8eafd5 feat(bot): Коммит — создание, замена и удаление файлов, черновик
- новый файл: имя → содержимое текстом (```-обёртка снимается) или документом
- новая папка (виртуальная, пока в ней нет файлов), замена с diff +N/−M, удаление
- валидация: имя, ≤ 100 КБ, UTF-8, без NUL; .py — ast.parse с выбором «исправить / всё равно»;
  .sourcecraft/, .tgit/, ruff.toml из бота не меняются
- DraftService: черновик в БД, 24 ч, ≤ 50 изменений; удаление нового файла убирает его из черновика
- дерево показывает черновик поверх репозитория: 🆕 / ✏️ / 🗑, размер — как будет после пуша
- экран черновика (A/M/D) с «↩️ убрать»

Refs #26

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 19:18:49 +03:00
ElebobaandClaude Opus 5.5 dbb17bd774 feat(bot): Коммит — навигация по дереву репозитория загрузок
- UploadsRepo: локальный клон, fetch не чаще fetch_interval, ls-tree -z (пробелы/кириллица),
  cat-file, последний коммит по пути; git без shell, ssh с -F /dev/null (без конфига хоста)
- «📝 Коммит»: папки/файлы с размерами, ⬆️ Вверх, страницы по 20, карточка файла
  с последним коммитом, «👁 Показать» (<pre> или документом); в кнопках только индексы
- ключ для контейнера: GIT_SSH_KEY_B64 → временный файл 0600 (смонтированный файл хоста
  контейнерному uid не прочитать); volume /data для клона
- тесты на настоящем bare-репозитории во временной папке

Refs #8

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 19:10:30 +03:00
ElebobaandClaude Opus 5.5 219bebd5a2 feat(bot): доступ по QR-приглашениям, раздел «Участники», отзыв доступа
- AccessService: одноразовые приглашения (128 бит, в БД только sha256, TTL 24 ч),
  атомарный захват приглашения (условный UPDATE — из двух сканов одного QR проходит один),
  антиперебор (> 5 неудач в час — молчание), отзыв (черновик и FSM удаляются), повторное приглашение
- ADMIN_IDS заводятся в БД при старте; дальше — только по приглашению
- AccessMiddleware: участники из БД (+ last_seen раз в 5 мин), новичкам — только /start <токен>
- «👥 Участники» для админов: список, QR (segno) + ссылка, отзыв с подтверждением
- один ответ на неверный/истёкший/использованный токен — без подсказок

Refs #25

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 19:03:50 +03:00
Eleboba 4e5930b4c0 Merge pull request 'feat(bot): клиент SourceCraft API' (#32) from feat/10-sourcecraft-client into dev
Reviewed-on: #32
2026-10-15 18:52:04 +03:00
ElebobaandClaude Opus 5.5 25735fee96 feat(bot): клиент SourceCraft API — прогоны, логи кубов, sha коммита, ретраи
- SourceCraftClient (httpx): list_runs, get_run, get_cube_logs (постранично), run_commit_sha;
  ретраи на сетевые ошибки и 5xx с экспоненциальной задержкой, 4xx — SourceCraftError без повторов
- модели: RunStatus/EventType из swagger, «1970-01-01» = не задано, failed_cubes без служебных #-кубов
- у прогона в API нет sha → куб check/meta в tgit-uploads печатает TGIT_COMMIT_SHA (ADR-014)
- CLI: python -m app.sourcecraft.cli runs
- тесты на реальных ответах API (фикстуры), порог покрытия 45 → 80

Refs #10

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 18:45:42 +03:00
ElebobaandClaude Opus 5.5 282920e06b feat(bot): модель данных, миграции Alembic, FSM в БД, TTL-чистка
- app/db/models.py: 13 таблиц (ARCHITECTURE §2.10); перечисления — VARCHAR + CHECK,
  JSON → JSONB на Postgres, BIGINT/INTEGER-ключи для SQLite
- миграция 0001 (автогенерация + ручная чистка: импорт sa.Text, дубли CHECK)
- миграции применяются при старте бота программно (app.db.migrate), script_location из пакета
- SqlStorage: FSM aiogram поверх fsm_state — состояние переживает рестарт (перенесено из #4)
- cleanup_loop: истёкшие черновики, снимки контейнеров и попытки входа старше суток
- тесты: upgrade/downgrade, дрейф моделей и миграций, CHECK, CASCADE, FSM, чистка

Refs #5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 18:45:16 +03:00
87 changed files with 7177 additions and 76 deletions
+11 -2
View File
@@ -5,6 +5,7 @@
TELEGRAM_BOT_TOKEN= # (secret) токен от @BotFather
ADMIN_IDS= # Telegram ID первых администраторов через запятую
LOG_LEVEL=INFO
TELEGRAM_PROXY_URL= # (secret) http://user:pass@host:port или socks5://… — на RF-сервере Telegram недоступен напрямую
# --- База данных ---
POSTGRES_PASSWORD=tgit # (secret на сервере) пароль Postgres в Compose
@@ -13,12 +14,20 @@ POSTGRES_PASSWORD=tgit # (secret на сервере) пароль Postg
# --- SourceCraft (#7, #10) ---
SOURCECRAFT_TOKEN= # (secret) персональный токен API
UPLOADS_REPO=rogachevegorushka/tgit-uploads
SOURCECRAFT_UPLOADS_REPO=rogachevegorushka/tgit-uploads
# SOURCECRAFT_RUN_URL_TEMPLATE=https://sourcecraft.dev/{repo}/cicd/runs/{run}
# --- Git: коммиты в репозиторий загрузок (#7, #9) ---
GIT_SSH_KEY_PATH= # путь к приватному deploy-ключу бота (монтируется read-only);
# публичная часть — deploy key с правом записи в tgit-uploads
UPLOADS_GIT_URL=git@ssh.sourcecraft.dev:rogachevegorushka/tgit-uploads.git
GIT_SSH_KEY_B64= # (secret) для Docker: тот же ключ в base64 — base64 -w0 ~/.ssh/tgit_bot_ed25519
GIT_UPLOADS_URL=git@ssh.sourcecraft.dev:rogachevegorushka/tgit-uploads.git
# GIT_CLONE_DIR=data/uploads # локальный клон репозитория загрузок (в Docker — volume /data/uploads)
# --- Docker: раздел «Статус» (#14) ---
# В Compose задан сам: http://docker-proxy:2375. Без него (make run) «Статус» отвечает «Docker недоступен».
# DOCKER_URL=
# DOCKER_LABEL=tgit.managed=true
# --- LLM (#17) ---
LLM_PROVIDER=mock # mock | openai_compat
+72
View File
@@ -0,0 +1,72 @@
name: CI
# Гейт качества на каждый PR и push в dev/main — как в Vobot (.gitea/workflows/backend.yml).
# CD (.gitea/workflows/deploy.yml) тесты не гоняет: они прошли здесь, на PR.
on:
push:
branches: [dev, main]
pull_request:
branches: [dev, main]
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
jobs:
lint-and-test:
# Имя джоба — контекст статуса в PR; если включать branch protection, требовать именно его.
name: ruff · mypy · pytest
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_DB: tgit_test
POSTGRES_USER: tgit
POSTGRES_PASSWORD: tgit
options: >-
--health-cmd pg_isready
--health-interval 5s
--health-timeout 5s
--health-retries 10
# без маппинга портов: act_runner запускает job в контейнере, сервис доступен по имени `postgres`
# (опыт Vobot: маппинг 5432 на хост конфликтует с postgres самого Gitea)
steps:
- uses: actions/checkout@v4
- name: Setup Python 3.12
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install uv
# pip, а не astral-sh/setup-uv: на self-hosted Gitea экшен ходит в GitHub API и падает (опыт Vobot)
run: pip install uv
- name: Install deps
run: uv sync --frozen
- name: Ruff
run: |
uv run ruff check .
uv run ruff format --check .
- name: Mypy
run: uv run mypy
- name: Migrations on PostgreSQL — up, down, up
# pytest гоняет схему на SQLite; здесь — настоящий Postgres: JSONB, BIGINT, CHECK, откат
env:
DATABASE_URL: postgresql+asyncpg://tgit:tgit@postgres:5432/tgit_test
run: |
uv run alembic upgrade head
uv run alembic downgrade base
uv run alembic upgrade head
- name: Pytest (+ порог покрытия из pyproject)
run: |
git config --global user.name ci
git config --global user.email ci@ci
uv run pytest -q --cov --cov-report=term
+45
View File
@@ -0,0 +1,45 @@
name: CD
# Как в Vobot: тесты здесь не гоняем — они прошли в CI на PR.
# dev → /srv/tgit/dev, main → /srv/tgit/prod на хосте Vobot (infra/deploy/deploy.sh).
# Окружение без /srv/tgit/<env>/.env на сервере пропускается (prod — когда будет свой токен бота).
on:
push:
branches: [dev, main]
workflow_dispatch: {}
# Деплои одной ветки — строго по очереди, без отмены.
concurrency:
group: cd-${{ github.ref }}
cancel-in-progress: false
jobs:
deploy:
name: deploy # Gitea doesn't render expressions in job names
runs-on: ubuntu-latest
env:
DEPLOY_ENV: ${{ github.ref_name == 'main' && 'prod' || 'dev' }}
steps:
- uses: actions/checkout@v4
# Серия мержей подряд: устаревший прогон сам сворачивается, задеплоит актуальный (опыт Vobot)
- id: guard
name: "Актуален ли ещё этот пуш?"
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then echo "go=true" >> "$GITHUB_OUTPUT"; exit 0; fi
REMOTE_SHA=$(git ls-remote origin "refs/heads/${{ github.ref_name }}" | cut -f1)
if [ "$REMOTE_SHA" = "${{ github.sha }}" ]; then echo "go=true" >> "$GITHUB_OUTPUT"
else echo "⏭ ${{ github.sha }} уже не HEAD ${{ github.ref_name }} — пропускаю"; echo "go=false" >> "$GITHUB_OUTPUT"; fi
- name: Deploy
if: steps.guard.outputs.go == 'true'
env:
# Через env:, а не инлайном в run: — многострочный ключ иначе может попасть в лог (опыт Vobot)
DEPLOY_SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
DEPLOY_HOST: ${{ secrets.DEPLOY_HOST }}
run: |
install -d -m 700 ~/.ssh
printf '%s\n' "$DEPLOY_SSH_KEY" > ~/.ssh/deploy_key
chmod 600 ~/.ssh/deploy_key
export DEPLOY_SSH_OPTS="-F /dev/null -i $HOME/.ssh/deploy_key -o IdentitiesOnly=yes -o StrictHostKeyChecking=accept-new -o ConnectTimeout=20"
infra/deploy/deploy.sh "$DEPLOY_ENV" HEAD
+4
View File
@@ -23,3 +23,7 @@ build/
.vscode/
.claude/worktrees/
.DS_Store
# local run without Docker: SQLite DB, uploads clone, demo repo
*.db
data/
+1
View File
@@ -49,6 +49,7 @@ Telegram-бот «git в телеграме» для команды (учебн
| [`docs/dev/ISSUE_TEMPLATE.md`](docs/dev/ISSUE_TEMPLATE.md) | Шаблон тела issue |
| [`docs/dev/GITEA_API.md`](docs/dev/GITEA_API.md) | Gitea API: issue, PR, зависимости, зеркало |
| [`docs/dev/SOURCECRAFT_API.md`](docs/dev/SOURCECRAFT_API.md) | SourceCraft API: CI-прогоны, логи, секреты |
| [`docs/dev/DEPLOY.md`](docs/dev/DEPLOY.md) | **Деплой:** сервер, окружения dev/prod, руками, откат |
---
+2
View File
@@ -19,6 +19,8 @@ ENV PATH="/opt/venv/bin:$PATH" PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1
COPY --from=build /opt/venv /opt/venv
WORKDIR /srv/tgit
COPY config ./config
# /data: the uploads repo clone (named volume inherits this ownership)
RUN install -d -o tgit -g tgit /data
USER tgit
# Liveness: the bot touches a heartbeat file while its event loop runs (app/health.py)
HEALTHCHECK --interval=15s --timeout=5s --start-period=20s --retries=3 CMD ["python", "-m", "app.health"]
+16 -1
View File
@@ -1,4 +1,4 @@
.PHONY: install lint fmt typecheck test cov up down logs ps
.PHONY: install lint fmt typecheck test cov up down logs ps run demo-repo
install:
uv sync
@@ -30,3 +30,18 @@ logs:
ps:
docker compose ps
# Run the bot on this machine without Docker: SQLite (./tgit.db) and the settings from .env
run:
uv run python -m app
# A local uploads repo seeded from the template — try the bot without SourceCraft:
# make demo-repo → put the printed GIT_UPLOADS_URL into .env → make run
DEMO_REPO := $(CURDIR)/data/demo-uploads.git
demo-repo:
@test ! -e $(DEMO_REPO) || { echo "$(DEMO_REPO) already exists"; exit 1; }
@rm -rf data/demo-seed && mkdir -p data && cp -r infra/uploads-template data/demo-seed
@cd data/demo-seed && git init -q -b main && git add -A \
&& git -c user.name=tgit-bot -c user.email=tgit-bot@users.noreply.tgit commit -qm "init: uploads template"
@git clone -q --bare data/demo-seed $(DEMO_REPO) && rm -rf data/demo-seed
@echo "GIT_UPLOADS_URL=$(DEMO_REPO)"
+5
View File
@@ -0,0 +1,5 @@
# Developer CLI: `uv run alembic revision --autogenerate -m "..."`. The DB URL comes from DATABASE_URL (.env),
# the bot itself applies migrations at startup (app/db/migrate.py).
[alembic]
script_location = app.db:migrations
file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s_%%(slug)s
View File
+13
View File
@@ -0,0 +1,13 @@
import io
import segno
def invite_link(bot_username: str, token: str) -> str:
return f"https://t.me/{bot_username}?start={token}"
def qr_png(data: str) -> bytes:
buf = io.BytesIO()
segno.make(data, error="m").save(buf, kind="png", scale=10, border=2)
return buf.getvalue()
+174
View File
@@ -0,0 +1,174 @@
"""Invites, members and revocation (SPEC §3.1, ADR-011)."""
import hashlib
import secrets
from collections.abc import Callable
from dataclasses import dataclass
from datetime import UTC, datetime, timedelta
from enum import StrEnum
from typing import Any, cast
from sqlalchemy import CursorResult, delete, func, select, update
from app.config import AccessSettings
from app.db.models import AccessAttempt, Draft, FsmState, Invite, Role, User, utcnow
from app.db.session import SessionMaker
LAST_SEEN_EVERY = timedelta(minutes=5)
ATTEMPT_WINDOW = timedelta(hours=1)
class RedeemStatus(StrEnum):
OK = "ok"
ALREADY_MEMBER = "already_member"
REJECTED = "rejected" # unknown, used or expired — one answer, no hints which tokens existed
THROTTLED = "throttled"
@dataclass(frozen=True)
class TgUser:
tg_id: int
full_name: str
username: str | None = None
@dataclass(frozen=True)
class CreatedInvite:
token: str
expires_at: datetime
def hash_token(token: str) -> str:
return hashlib.sha256(token.encode()).hexdigest()
class AccessService:
def __init__(self, sessions: SessionMaker, settings: AccessSettings, clock: Callable[[], datetime] = utcnow):
self._sessions = sessions
self._settings = settings
self._clock = clock
async def bootstrap_admins(self) -> None:
"""ADMIN_IDS become active admins in the DB; everybody else gets in only by invite (SPEC A-8)."""
async with self._sessions.begin() as s:
for tg_id in self._settings.admin_ids:
user = await s.get(User, tg_id)
if user is None:
s.add(User(tg_id=tg_id, full_name=f"admin {tg_id}", role=Role.ADMIN, joined_at=self._clock()))
else:
user.role, user.is_active = Role.ADMIN, True
async def get_member(self, tg_id: int) -> User | None:
async with self._sessions() as s:
user = await s.get(User, tg_id)
return user if user is not None and user.is_active else None
async def touch(self, member: User, seen: TgUser) -> None:
"""Refresh name and last-seen, at most every few minutes — not a write per update."""
now = self._clock()
last = member.last_seen_at
if (
last is not None
and now - _aware(last) < LAST_SEEN_EVERY
and (member.full_name, member.username) == (seen.full_name, seen.username)
):
return
async with self._sessions.begin() as s:
await s.execute(
update(User)
.where(User.tg_id == member.tg_id)
.values(last_seen_at=now, full_name=seen.full_name, username=seen.username)
)
async def create_invite(self, admin_id: int) -> CreatedInvite:
token = secrets.token_urlsafe(16) # 128 bits, 22 chars of [A-Za-z0-9_-]: fits the 64-char deep link payload
expires_at = self._clock() + timedelta(hours=self._settings.invite_ttl_hours)
async with self._sessions.begin() as s:
s.add(
Invite(
token_hash=hash_token(token), created_by=admin_id, created_at=self._clock(), expires_at=expires_at
)
)
return CreatedInvite(token=token, expires_at=expires_at)
async def redeem(self, token: str, who: TgUser) -> RedeemStatus:
now = self._clock()
try:
async with self._sessions.begin() as s:
existing = await s.get(User, who.tg_id)
if existing is not None and existing.is_active:
return RedeemStatus.ALREADY_MEMBER
failed = await s.scalar(
select(func.count())
.select_from(AccessAttempt)
.where(
AccessAttempt.tg_id == who.tg_id,
AccessAttempt.ok.is_(False),
AccessAttempt.at > now - ATTEMPT_WINDOW,
)
)
if (failed or 0) >= self._settings.max_failed_attempts:
return RedeemStatus.THROTTLED
invite = await s.scalar(select(Invite).where(Invite.token_hash == hash_token(token)))
if invite is None or invite.used_by is not None or _aware(invite.expires_at) <= now:
raise _Rejected
if existing is None:
s.add(
User(
tg_id=who.tg_id,
full_name=who.full_name,
username=who.username,
role=Role.MEMBER,
invited_by=invite.created_by,
joined_at=now,
last_seen_at=now,
)
)
else: # revoked earlier, invited again
existing.is_active, existing.role, existing.invited_by = True, Role.MEMBER, invite.created_by
existing.full_name, existing.username, existing.last_seen_at = who.full_name, who.username, now
await s.flush() # the user row must exist before invites.used_by points at it
# Claim atomically: of two people scanning the same QR only one UPDATE matches the row,
# the loser's transaction (with its freshly created user) is rolled back.
result = await s.execute(
update(Invite)
.where(Invite.id == invite.id, Invite.used_by.is_(None))
.values(used_by=who.tg_id, used_at=now)
)
if cast(CursorResult[Any], result).rowcount != 1:
raise _Rejected
s.add(AccessAttempt(tg_id=who.tg_id, at=now, ok=True))
except _Rejected:
async with self._sessions.begin() as s:
s.add(AccessAttempt(tg_id=who.tg_id, at=now, ok=False))
return RedeemStatus.REJECTED
return RedeemStatus.OK
async def list_members(self) -> list[User]:
async with self._sessions() as s:
rows = await s.scalars(select(User).where(User.is_active.is_(True)).order_by(User.role, User.joined_at))
return list(rows)
async def revoke(self, tg_id: int) -> bool:
"""Cut a member off at once: their updates are ignored from now on, draft and dialog state are dropped."""
async with self._sessions.begin() as s:
user = await s.get(User, tg_id)
if user is None or not user.is_active or user.role is Role.ADMIN:
return False
user.is_active = False
await s.execute(delete(Draft).where(Draft.user_id == tg_id))
# FSM keys are "bot:chat:user:…"; in a private chat chat == user
await s.execute(delete(FsmState).where(FsmState.key.like(f"%:{tg_id}:{tg_id}:%")))
return True
class _Rejected(Exception):
pass
def _aware(value: datetime) -> datetime:
# SQLite hands back naive datetimes even for timezone=True columns; everything we store is UTC.
return value if value.tzinfo is not None else value.replace(tzinfo=UTC)
+6 -4
View File
@@ -1,8 +1,10 @@
"""FSM storage factory. In-memory for now; Postgres-backed storage that survives restarts arrives with the DB (#5)."""
"""FSM storage factory: dialog state lives in the DB and survives restarts (SPEC M-4)."""
from aiogram.fsm.storage.base import BaseStorage
from aiogram.fsm.storage.memory import MemoryStorage
from app.bot.fsm_storage import SqlStorage
from app.db.session import SessionMaker
def make_storage() -> BaseStorage:
return MemoryStorage()
def make_storage(sessions: SessionMaker) -> BaseStorage:
return SqlStorage(sessions)
+54
View File
@@ -0,0 +1,54 @@
"""aiogram FSM storage on top of the `fsm_state` table — dialog state survives bot restarts (SPEC M-4)."""
from collections.abc import Mapping
from typing import Any
from aiogram.fsm.state import State
from aiogram.fsm.storage.base import BaseStorage, StateType, StorageKey
from app.db.models import FsmState
from app.db.session import SessionMaker
def key_to_str(key: StorageKey) -> str:
parts = (key.bot_id, key.chat_id, key.user_id, key.thread_id, key.business_connection_id, key.destiny)
return ":".join("" if p is None else str(p) for p in parts)
class SqlStorage(BaseStorage):
def __init__(self, sessions: SessionMaker) -> None:
self._sessions = sessions
async def set_state(self, key: StorageKey, state: StateType = None) -> None:
value = state.state if isinstance(state, State) else state
await self._write(key, state=value)
async def get_state(self, key: StorageKey) -> str | None:
async with self._sessions() as session:
row = await session.get(FsmState, key_to_str(key))
return row.state if row else None
async def set_data(self, key: StorageKey, data: Mapping[str, Any]) -> None:
await self._write(key, data=dict(data))
async def get_data(self, key: StorageKey) -> dict[str, Any]:
async with self._sessions() as session:
row = await session.get(FsmState, key_to_str(key))
return dict(row.data) if row else {}
async def close(self) -> None:
pass # the engine is owned and disposed by the app
async def _write(self, key: StorageKey, **fields: Any) -> None:
async with self._sessions.begin() as session:
row = await session.get(FsmState, key_to_str(key))
state = fields.get("state", row.state if row else None)
data = fields.get("data", row.data if row else {})
if state is None and not data:
if row is not None:
await session.delete(row) # empty state: keep the table small
return
if row is None:
session.add(FsmState(key=key_to_str(key), state=state, data=data))
else:
row.state, row.data = state, data
+28 -10
View File
@@ -1,6 +1,7 @@
"""Access gate on every update — messages and button presses alike (SPEC A-5).
"""Access gate on every update — messages and button presses alike (SPEC A-5, ADR-011).
Until invites land (#25) the only members are the bootstrap admins from `ADMIN_IDS`.
Members (active users in the DB) pass with `member` injected into handler data. Outsiders pass only with
`/start <token>` — the invite deep link; everything else gets one "invite only" reply a day, then silence.
"""
import time
@@ -8,18 +9,28 @@ from collections.abc import Awaitable, Callable
from typing import Any
from aiogram import BaseMiddleware
from aiogram.types import CallbackQuery, Message, TelegramObject
from aiogram.types import CallbackQuery, Message, TelegramObject, User
from app.access.service import AccessService, TgUser
from app.texts import Texts
IsMember = Callable[[int], Awaitable[bool]]
NO_ACCESS_REPLY_EVERY = 24 * 3600.0
def tg_user(user: User) -> TgUser:
return TgUser(tg_id=user.id, full_name=user.full_name, username=user.username)
def is_invite_link(event: TelegramObject) -> bool:
if not isinstance(event, Message) or not event.text:
return False
parts = event.text.split(maxsplit=1)
return len(parts) == 2 and parts[0].split("@")[0] == "/start"
class AccessMiddleware(BaseMiddleware):
def __init__(self, is_member: IsMember, texts: Texts, clock: Callable[[], float] = time.monotonic) -> None:
self._is_member = is_member
def __init__(self, access: AccessService, texts: Texts, clock: Callable[[], float] = time.monotonic) -> None:
self._access = access
self._texts = texts
self._clock = clock
# tg_id -> when we last told this outsider "invite only"; answering every message would let them spam us.
@@ -31,10 +42,17 @@ class AccessMiddleware(BaseMiddleware):
event: TelegramObject,
data: dict[str, Any],
) -> Any:
user = data.get("event_from_user")
if user is not None and await self._is_member(user.id):
user: User | None = data.get("event_from_user")
if user is None:
return None
member = await self._access.get_member(user.id)
if member is not None:
await self._access.touch(member, tg_user(user))
data["member"] = member
return await handler(event, data)
if is_invite_link(event):
data["member"] = None
return await handler(event, data)
if user is not None:
await self._refuse(event, user.id)
return None
+611
View File
@@ -0,0 +1,611 @@
"""«📝 Коммит»: browse the uploads repo tree and collect changes into a draft (SPEC C-1…C-16). Push — #9."""
import base64
import html
import io
import logging
from dataclasses import dataclass
from enum import StrEnum
from typing import Any
from aiogram import Bot, F, Router
from aiogram.enums import ParseMode
from aiogram.filters.callback_data import CallbackData
from aiogram.fsm.context import FSMContext
from aiogram.fsm.state import State, StatesGroup
from aiogram.types import BufferedInputFile, CallbackQuery, InlineKeyboardMarkup, Message
from aiogram.utils.keyboard import InlineKeyboardBuilder
from app.bot.menu import Nav, NavAction
from app.db.models import ChangeOp, User
from app.repo.drafts import DraftFull, DraftService, DraftView
from app.repo.git import Entry, GitError, UploadsRepo
from app.repo.publisher import Publisher, PushResult, PushStatus
from app.repo.validate import (
MAX_FILE_BYTES,
ValidationError,
check_content,
check_message,
check_name,
diff_stat,
is_protected,
join,
python_syntax,
strip_fence,
)
from app.texts import CommitTexts, Texts
log = logging.getLogger(__name__)
PAGE_SIZE = 20
INLINE_VIEW_LIMIT = 3500 # chars; Telegram caps a message at 4096 and <pre> markup takes some
class Act(StrEnum):
OPEN = "o"
UP = "u"
PAGE = "p"
VIEW = "v"
TO_DIR = "b"
REPLACE = "r"
DELETE = "d"
NEW_FILE = "f"
NEW_DIR = "n"
DRAFT = "dr"
UNSTAGE = "x"
FORCE = "y"
FIX = "z"
MESSAGE = "m"
PUSH = "ps"
PUSH_GO = "pg"
PUSH_FORCE = "pf"
COMMIT = "cm"
class CommitCb(CallbackData, prefix="c"):
# Only an action and a small index: paths never go into callback_data (64-byte limit, ARCHITECTURE §2.1)
a: Act
i: int = 0
class Flow(StatesGroup):
file_name = State()
dir_name = State()
content = State()
syntax = State()
message = State()
class Mark(StrEnum):
NONE = ""
NEW = "new"
MODIFIED = "mod"
DELETED = "del"
@dataclass(frozen=True)
class Row:
"""A line of the directory screen: a repo entry with the draft laid over it."""
path: str
kind: str
size: int
mark: Mark
@property
def name(self) -> str:
return self.path.rsplit("/", 1)[-1]
def human_size(n: int) -> str:
size = float(n)
for unit in ("Б", "КБ", "МБ", "ГБ"):
if size < 1024 or unit == "ГБ":
return f"{size:.0f} {unit}" if unit == "Б" else f"{size:.1f} {unit}"
size /= 1024
raise AssertionError("unreachable")
def parent(path: str) -> str:
return path.rsplit("/", 1)[0] if "/" in path else ""
def overlay(path: str, entries: list[Entry], draft: DraftView | None, virtual_dirs: list[str]) -> list[Row]:
"""Repo listing + what the draft adds, changes or deletes in this directory."""
changes = {c.path: c for c in draft.changes} if draft else {}
rows: dict[str, Row] = {}
for e in entries:
change = changes.get(e.path)
mark, size = Mark.NONE, e.size
if change is not None:
mark = Mark.DELETED if change.op is ChangeOp.DELETE else Mark.MODIFIED
if change.content is not None:
size = len(change.content) # show the size the file will have after the push
rows[e.path] = Row(e.path, e.kind, size, mark)
prefix = f"{path}/" if path else ""
for change in changes.values():
if change.op is not ChangeOp.ADD or not change.path.startswith(prefix):
continue
rest = change.path[len(prefix) :]
if "/" in rest: # a file deeper down: show its top folder here
sub = prefix + rest.split("/", 1)[0]
rows.setdefault(sub, Row(sub, "dir", 0, Mark.NEW))
else:
rows[change.path] = Row(change.path, "file", len(change.content or b""), Mark.NEW)
for vd in virtual_dirs:
if parent(vd) == path:
rows.setdefault(vd, Row(vd, "dir", 0, Mark.NEW))
return sorted(rows.values(), key=lambda r: (r.kind != "dir", r.name.lower()))
def mark_prefix(t: CommitTexts, mark: Mark) -> str:
return {Mark.NONE: "", Mark.NEW: t.mark_new, Mark.MODIFIED: t.mark_modified, Mark.DELETED: t.mark_deleted}[mark]
def render_dir(
texts: Texts, path: str, rows: list[Row], page: int, draft_count: int
) -> tuple[str, InlineKeyboardMarkup]:
t = texts.commit
pages = max(1, -(-len(rows) // PAGE_SIZE))
page = min(max(page, 0), pages - 1)
lines = [t.title.format(path=path)]
if not rows:
lines.append(t.empty)
elif pages > 1:
lines.append(t.page.format(page=page + 1, pages=pages))
kb = InlineKeyboardBuilder()
start = page * PAGE_SIZE
shown = rows[start : start + PAGE_SIZE]
for index, r in enumerate(shown, start=start):
base = (
t.dir_button.format(name=r.name)
if r.kind == "dir"
else t.file_button.format(name=r.name, size=human_size(r.size))
)
kb.button(text=mark_prefix(t, r.mark) + base, callback_data=CommitCb(a=Act.OPEN, i=index))
sizes = [1] * len(shown)
paging: list[tuple[str, CommitCb]] = []
if page > 0:
paging.append((t.prev, CommitCb(a=Act.PAGE, i=page - 1)))
if page < pages - 1:
paging.append((t.next, CommitCb(a=Act.PAGE, i=page + 1)))
for text, cb in paging:
kb.button(text=text, callback_data=cb)
if paging:
sizes.append(len(paging))
kb.button(text=t.new_file, callback_data=CommitCb(a=Act.NEW_FILE))
kb.button(text=t.new_dir, callback_data=CommitCb(a=Act.NEW_DIR))
sizes.append(2)
if draft_count:
# the main action once something is staged — right in the folder, not two screens deep
kb.button(text=t.commit_button.format(count=draft_count), callback_data=CommitCb(a=Act.COMMIT))
kb.button(text=t.draft_button.format(count=draft_count), callback_data=CommitCb(a=Act.DRAFT))
sizes += [1, 1]
if path:
kb.button(text=t.up, callback_data=CommitCb(a=Act.UP))
sizes.append(1)
kb.button(text=texts.menu.back, callback_data=Nav(action=NavAction.BACK))
kb.button(text=texts.menu.cancel, callback_data=Nav(action=NavAction.CANCEL))
sizes.append(2)
kb.adjust(*sizes)
return "\n".join(lines), kb.as_markup()
def file_keyboard(texts: Texts) -> InlineKeyboardMarkup:
t = texts.commit
kb = InlineKeyboardBuilder()
kb.button(text=t.view, callback_data=CommitCb(a=Act.VIEW))
kb.button(text=t.replace, callback_data=CommitCb(a=Act.REPLACE))
kb.button(text=t.delete, callback_data=CommitCb(a=Act.DELETE))
kb.button(text=t.to_dir, callback_data=CommitCb(a=Act.TO_DIR))
kb.adjust(1, 2, 1)
return kb.as_markup()
def to_dir_keyboard(texts: Texts) -> InlineKeyboardMarkup:
kb = InlineKeyboardBuilder()
kb.button(text=texts.commit.to_dir, callback_data=CommitCb(a=Act.TO_DIR))
return kb.as_markup()
def build_router(
texts: Texts, repo: UploadsRepo, drafts: DraftService, publisher: Publisher, commit_url_template: str = ""
) -> Router:
router = Router(name="commit")
t = texts.commit
async def dir_screen(state: FSMContext, user_id: int, path: str, page: int) -> tuple[str, InlineKeyboardMarkup]:
data = await state.get_data()
virtual: list[str] = data.get("commit_virtual", [])
draft = await drafts.get(user_id)
try:
entries = await repo.list_dir(path)
except GitError:
# a folder that exists only in the draft (or a virtual one) has nothing in the repo yet
if path not in virtual and not (draft and any(c.path.startswith(f"{path}/") for c in draft.changes)):
raise
entries = []
rows = overlay(path, entries, draft, virtual)
await state.update_data(
commit_path=path,
commit_page=page,
commit_rows=[[r.path, r.kind, r.size, r.mark.value] for r in rows],
commit_file=None,
)
return render_dir(texts, path, rows, page, len(draft.changes) if draft else 0)
async def draft_screen(state: FSMContext, user_id: int) -> tuple[str, InlineKeyboardMarkup]:
draft = await drafts.get(user_id)
changes = draft.changes if draft else []
await state.update_data(commit_draft=[c.path for c in changes])
kb = InlineKeyboardBuilder()
if not changes:
text = t.draft_empty
else:
ops = {ChangeOp.ADD: t.op_add, ChangeOp.MODIFY: t.op_modify, ChangeOp.DELETE: t.op_delete}
lines = [t.draft_title.format(count=len(changes)), ""]
lines += [f"{ops[c.op]} /{c.path}" for c in changes]
message = draft.message if draft else None
lines += ["", t.message_current.format(message=message) if message else t.message_missing]
text = "\n".join(lines)
for i, c in enumerate(changes):
kb.button(
text=t.unstage.format(name=c.path.rsplit("/", 1)[-1]), callback_data=CommitCb(a=Act.UNSTAGE, i=i)
)
if changes:
kb.button(text=t.message_button, callback_data=CommitCb(a=Act.MESSAGE))
if draft and draft.message:
kb.button(text=t.push_button, callback_data=CommitCb(a=Act.PUSH))
kb.button(text=t.to_dir, callback_data=CommitCb(a=Act.TO_DIR))
kb.adjust(1)
return text, kb.as_markup()
def confirm_screen(draft: DraftView) -> tuple[str, InlineKeyboardMarkup]:
builder = InlineKeyboardBuilder()
builder.button(text=t.push_go, callback_data=CommitCb(a=Act.PUSH_GO))
builder.button(text=t.edit_message, callback_data=CommitCb(a=Act.MESSAGE))
builder.button(text=texts.menu.back, callback_data=CommitCb(a=Act.TO_DIR))
builder.adjust(1)
return t.push_confirm.format(summary=summary(draft), message=draft.message), builder.as_markup()
def summary(draft: DraftView) -> str:
ops = {ChangeOp.ADD: t.op_add, ChangeOp.MODIFY: t.op_modify, ChangeOp.DELETE: t.op_delete}
return "\n".join(f"{ops[c.op]} /{c.path}" for c in draft.changes)
async def do_push(msg: Message, state: FSMContext, member: User, *, overwrite: bool) -> None:
draft = await drafts.get(member.tg_id)
if draft is None or not draft.changes or not draft.message:
text, kb = await draft_screen(state, member.tg_id)
await msg.edit_text(text, reply_markup=kb)
return
await msg.edit_text(t.pushing)
result: PushResult = await publisher.push(member, draft, draft.message, overwrite=overwrite)
match result.status:
case PushStatus.OK:
assert result.sha is not None
first_line = draft.message.splitlines()[0]
lines = [t.pushed.format(sha=result.sha[:7], message=first_line)]
if commit_url_template:
lines.append(commit_url_template.format(sha=result.sha))
lines.append(t.ci_pending)
await msg.edit_text("\n".join(lines))
await publisher.record(member, result, draft.message, draft, msg.chat.id, msg.message_id)
await drafts.clear(member.tg_id)
await state.update_data(commit_virtual=[])
case PushStatus.CONFLICT:
builder = InlineKeyboardBuilder()
builder.button(text=t.overwrite, callback_data=CommitCb(a=Act.PUSH_FORCE))
builder.button(text=texts.menu.cancel, callback_data=CommitCb(a=Act.DRAFT))
paths = "\n".join(f"/{p}" for p in result.conflicts)
await msg.edit_text(t.conflict.format(paths=paths), reply_markup=builder.as_markup())
case PushStatus.EMPTY:
await drafts.clear(member.tg_id)
await msg.edit_text(t.push_empty)
case PushStatus.LIMIT:
await msg.edit_text(t.push_limit, reply_markup=to_dir_keyboard(texts))
case PushStatus.FAILED:
await msg.edit_text(t.push_failed, reply_markup=to_dir_keyboard(texts))
async def file_card(user_id: int, path: str, size: int, mark: Mark) -> str:
card = t.file_card.format(path=path, size=human_size(size))
if mark is Mark.NEW:
return f"{card}\n{t.in_draft_new}"
info = await repo.last_commit(path)
tail = (
t.last_commit.format(
sha=info.sha[:7], author=info.author, date=info.date.strftime("%d.%m.%Y %H:%M"), subject=info.subject
)
if info
else t.no_commits
)
note = {Mark.MODIFIED: f"\n{t.in_draft_modified}", Mark.DELETED: f"\n{t.in_draft_deleted}"}.get(mark, "")
return f"{card}\n{tail}{note}"
async def current_content(user_id: int, path: str) -> bytes | None:
"""What the member sees as the file now: the draft version if there is one."""
draft = await drafts.get(user_id)
change = draft.change_for(path) if draft else None
if change is not None:
return change.content # None for a deleted file
return await repo.read(path)
async def send_file(msg: Message, user_id: int, path: str) -> None:
content = await current_content(user_id, path)
if content is None:
return
try:
text = content.decode("utf-8")
except UnicodeDecodeError:
text = None
if text is not None and "\0" not in text and len(text) <= INLINE_VIEW_LIMIT:
await msg.answer(f"<b>/{html.escape(path)}</b>\n<pre>{html.escape(text)}</pre>", parse_mode=ParseMode.HTML)
else:
await msg.answer_document(BufferedInputFile(content, path.rsplit("/", 1)[-1]), caption=t.sent_as_file)
async def stage(message: Message, state: FSMContext, member: User, path: str, op: ChangeOp, content: bytes) -> None:
old = await current_content(member.tg_id, path) if op is ChangeOp.MODIFY else None
try:
await drafts.put(member.tg_id, await repo.head(), path, op, content)
except DraftFull:
await message.answer(t.draft_full)
return
if op is ChangeOp.ADD:
note = t.added.format(path=path)
else:
added, removed = diff_stat((old or b"").decode("utf-8", errors="replace"), content.decode("utf-8"))
note = t.modified.format(path=path, added=added, removed=removed)
data = await state.get_data()
# a virtual folder becomes real as soon as a file lands in it
virtual = [v for v in data.get("commit_virtual", []) if not path.startswith(f"{v}/")]
await state.set_state(None)
await state.update_data(commit_virtual=virtual, commit_pending=None)
text, kb = await dir_screen(state, member.tg_id, data.get("commit_path", ""), 0)
await message.answer(f"{note}\n\n{text}", reply_markup=kb)
# ---- entry point -------------------------------------------------------------------------------------------
@router.message(F.text == texts.menu.commit)
async def open_commit(message: Message, state: FSMContext, member: User) -> None:
await state.clear()
try:
await repo.refresh()
text, kb = await dir_screen(state, member.tg_id, "", 0)
except GitError:
log.exception("uploads repo unavailable")
await message.answer(t.unavailable)
return
await message.answer(text, reply_markup=kb)
# ---- buttons -----------------------------------------------------------------------------------------------
@router.callback_query(CommitCb.filter())
async def on_action(query: CallbackQuery, callback_data: CommitCb, state: FSMContext, member: User) -> None:
if not isinstance(query.message, Message):
await query.answer()
return
msg = query.message
data: dict[str, Any] = await state.get_data()
path: str = data.get("commit_path", "")
rows: list[list[Any]] = data.get("commit_rows", [])
uid = member.tg_id
try:
match callback_data.a:
case Act.OPEN:
if not 0 <= callback_data.i < len(rows):
await query.answer() # stale button from an older screen
return
target, kind, size, mark = rows[callback_data.i]
if kind == "dir":
await repo.refresh()
text, kb = await dir_screen(state, uid, target, 0)
await msg.edit_text(text, reply_markup=kb)
else:
await state.update_data(commit_file=target)
await msg.edit_text(
await file_card(uid, target, size, Mark(mark)), reply_markup=file_keyboard(texts)
)
case Act.UP:
text, kb = await dir_screen(state, uid, parent(path), 0)
await msg.edit_text(text, reply_markup=kb)
case Act.PAGE:
real = [Row(p, k, s, Mark(m)) for p, k, s, m in rows]
draft = await drafts.get(uid)
await state.update_data(commit_page=callback_data.i)
text, kb = render_dir(texts, path, real, callback_data.i, len(draft.changes) if draft else 0)
await msg.edit_text(text, reply_markup=kb)
case Act.TO_DIR:
await state.set_state(None)
text, kb = await dir_screen(state, uid, path, data.get("commit_page", 0))
await msg.edit_text(text, reply_markup=kb)
case Act.VIEW:
if data.get("commit_file"):
await send_file(msg, uid, data["commit_file"])
case Act.NEW_FILE:
await state.set_state(Flow.file_name)
await msg.edit_text(t.ask_file_name.format(dir=path), reply_markup=to_dir_keyboard(texts))
case Act.NEW_DIR:
await state.set_state(Flow.dir_name)
await msg.edit_text(t.ask_dir_name.format(dir=path), reply_markup=to_dir_keyboard(texts))
case Act.REPLACE:
target = data.get("commit_file")
if not target:
await query.answer()
return
if is_protected(target):
await query.answer(t.protected, show_alert=True)
return
await state.set_state(Flow.content)
await state.update_data(commit_target=target, commit_op=ChangeOp.MODIFY.value)
await msg.edit_text(t.ask_content.format(path=target), reply_markup=to_dir_keyboard(texts))
case Act.DELETE:
target = data.get("commit_file")
if not target:
await query.answer()
return
if is_protected(target):
await query.answer(t.protected, show_alert=True)
return
await drafts.put(uid, await repo.head(), target, ChangeOp.DELETE, None)
text, kb = await dir_screen(state, uid, path, data.get("commit_page", 0))
await msg.edit_text(f"{t.deleted.format(path=target)}\n\n{text}", reply_markup=kb)
case Act.DRAFT:
text, kb = await draft_screen(state, uid)
await msg.edit_text(text, reply_markup=kb)
case Act.UNSTAGE:
paths: list[str] = data.get("commit_draft", [])
if 0 <= callback_data.i < len(paths):
await drafts.drop(uid, paths[callback_data.i])
text, kb = await draft_screen(state, uid)
await msg.edit_text(text, reply_markup=kb)
case Act.FORCE:
pending = data.get("commit_pending")
if pending is None or await state.get_state() != Flow.syntax.state:
await query.answer()
return
await query.answer()
await stage(
msg,
state,
member,
data["commit_target"],
ChangeOp(data["commit_op"]),
base64.b64decode(pending),
)
return
case Act.MESSAGE:
await state.set_state(Flow.message)
await state.update_data(commit_then_push=False)
await msg.edit_text(t.ask_message, reply_markup=to_dir_keyboard(texts))
case Act.PUSH:
draft = await drafts.get(uid)
if draft is None or not draft.changes or not draft.message:
await query.answer()
return
confirm, markup = confirm_screen(draft)
await msg.edit_text(confirm, reply_markup=markup)
case Act.COMMIT:
draft = await drafts.get(uid)
if draft is None or not draft.changes:
await query.answer()
return
if draft.message:
confirm, markup = confirm_screen(draft)
await msg.edit_text(confirm, reply_markup=markup)
else:
# ask for the message, then go straight to the push confirmation
await state.set_state(Flow.message)
await state.update_data(commit_then_push=True)
await msg.edit_text(t.ask_message, reply_markup=to_dir_keyboard(texts))
case Act.PUSH_GO | Act.PUSH_FORCE:
await query.answer()
await do_push(msg, state, member, overwrite=callback_data.a is Act.PUSH_FORCE)
return
case Act.FIX:
await state.set_state(Flow.content)
await state.update_data(commit_pending=None)
await msg.edit_text(
t.ask_content.format(path=data.get("commit_target", "")), reply_markup=to_dir_keyboard(texts)
)
except GitError:
log.exception("uploads repo unavailable")
await query.answer(t.unavailable, show_alert=True)
return
except DraftFull:
await query.answer(t.draft_full, show_alert=True)
return
await query.answer()
# ---- text input --------------------------------------------------------------------------------------------
@router.message(Flow.file_name, F.text)
async def got_file_name(message: Message, state: FSMContext, member: User) -> None:
data = await state.get_data()
directory = data.get("commit_path", "")
try:
name = check_name(message.text or "")
except ValidationError:
await message.answer(t.bad_name)
return
path = join(directory, name)
if is_protected(path):
await message.answer(t.protected)
return
taken = {row[0] for row in data.get("commit_rows", [])}
if path in taken:
await message.answer(t.exists)
return
await state.set_state(Flow.content)
await state.update_data(commit_target=path, commit_op=ChangeOp.ADD.value)
await message.answer(t.ask_content.format(path=path), reply_markup=to_dir_keyboard(texts))
@router.message(Flow.dir_name, F.text)
async def got_dir_name(message: Message, state: FSMContext, member: User) -> None:
data = await state.get_data()
directory = data.get("commit_path", "")
try:
name = check_name(message.text or "")
except ValidationError:
await message.answer(t.bad_name)
return
path = join(directory, name)
if is_protected(path + "/x"):
await message.answer(t.protected)
return
if path in {row[0] for row in data.get("commit_rows", [])}:
await message.answer(t.exists)
return
await state.set_state(None)
await state.update_data(commit_virtual=[*data.get("commit_virtual", []), path])
text, kb = await dir_screen(state, member.tg_id, path, 0)
await message.answer(text, reply_markup=kb)
@router.message(Flow.message, F.text)
async def got_message(message: Message, state: FSMContext, member: User) -> None:
try:
text = check_message(message.text or "")
except ValidationError:
await message.answer(t.bad_message)
return
await drafts.set_message(member.tg_id, text)
await state.set_state(None)
draft = await drafts.get(member.tg_id)
if (await state.get_data()).get("commit_then_push") and draft is not None and draft.changes:
await state.update_data(commit_then_push=False)
screen, kb = confirm_screen(draft)
else:
screen, kb = await draft_screen(state, member.tg_id)
await message.answer(screen, reply_markup=kb)
@router.message(Flow.content, F.text | F.document)
async def got_content(message: Message, state: FSMContext, member: User, bot: Bot) -> None:
data = await state.get_data()
target: str = data["commit_target"]
if message.document is not None:
if (message.document.file_size or 0) > MAX_FILE_BYTES:
await message.answer(t.too_big)
return
buf = io.BytesIO()
await bot.download(message.document, destination=buf)
raw = buf.getvalue()
else:
raw = strip_fence(message.text or "").encode("utf-8")
try:
text = check_content(raw)
except ValidationError as e:
await message.answer({"too_big": t.too_big, "binary": t.binary, "encoding": t.encoding}[str(e)])
return
issue = python_syntax(target, text)
if issue is not None:
await state.set_state(Flow.syntax)
await state.update_data(commit_pending=base64.b64encode(raw).decode())
kb = InlineKeyboardBuilder()
kb.button(text=t.fix, callback_data=CommitCb(a=Act.FIX))
kb.button(text=t.force, callback_data=CommitCb(a=Act.FORCE))
await message.answer(t.syntax.format(line=issue.line, message=issue.message), reply_markup=kb.as_markup())
return
await stage(message, state, member, target, ChangeOp(data["commit_op"]), raw)
return router
+26 -18
View File
@@ -1,49 +1,57 @@
"""/start, /help, /menu, main menu buttons and in-section navigation."""
"""/start (incl. invite deep links), /help, /menu, main menu buttons and in-section navigation."""
from aiogram import F, Router
from aiogram.filters import Command, CommandStart
from aiogram.filters import Command, CommandObject, CommandStart
from aiogram.fsm.context import FSMContext
from aiogram.types import CallbackQuery, Message
from app.access.service import AccessService, RedeemStatus
from app.bot.menu import Nav, NavAction, Section, main_menu, section_by_button, section_nav
from app.bot.middlewares.access import tg_user
from app.db.models import Role, User
from app.texts import Texts
def build_router(texts: Texts, admin_ids: frozenset[int]) -> Router:
def build_router(texts: Texts, access: AccessService) -> Router:
router = Router(name="common")
sections = section_by_button(texts)
section_text = {
Section.COMMIT: texts.section.commit,
Section.STATUS: texts.section.status,
Section.LOGS: texts.section.logs,
Section.ALERTS: texts.section.alerts,
Section.MEMBERS: texts.section.members,
}
def is_admin(message: Message) -> bool:
return message.from_user is not None and message.from_user.id in admin_ids
@router.message(CommandStart())
async def start(message: Message, state: FSMContext) -> None:
async def start(message: Message, command: CommandObject, state: FSMContext, member: User | None) -> None:
if member is None:
# Only an outsider with an invite token gets here (AccessMiddleware); a member's /start <x> is a plain start
if message.from_user is None or not command.args:
return
status = await access.redeem(command.args, tg_user(message.from_user))
if status is RedeemStatus.THROTTLED:
return # brute force: stay silent (SPEC A-6)
if status is RedeemStatus.REJECTED:
await message.answer(texts.start.invite_rejected)
return
member = await access.get_member(message.from_user.id)
if member is None:
return
await state.clear()
await message.answer(texts.start.welcome, reply_markup=main_menu(texts, is_admin=is_admin(message)))
await message.answer(texts.start.welcome, reply_markup=main_menu(texts, is_admin=member.role is Role.ADMIN))
@router.message(Command("menu"))
async def menu(message: Message, state: FSMContext) -> None:
async def menu(message: Message, state: FSMContext, member: User) -> None:
await state.clear()
await message.answer(texts.menu.placeholder, reply_markup=main_menu(texts, is_admin=is_admin(message)))
await message.answer(texts.menu.placeholder, reply_markup=main_menu(texts, is_admin=member.role is Role.ADMIN))
@router.message(Command("help"))
async def help_(message: Message) -> None:
await message.answer(texts.help.text)
@router.message(F.text.in_(sections))
# Sections with their own routers (members, commit) are matched there first
@router.message(F.text.in_({k for k, v in sections.items() if v in section_text}))
async def open_section(message: Message, state: FSMContext) -> None:
section = sections[message.text or ""]
if section is Section.MEMBERS and not is_admin(message):
return
await state.clear()
await message.answer(section_text[section], reply_markup=section_nav(texts))
await message.answer(section_text[sections[message.text or ""]], reply_markup=section_nav(texts))
@router.callback_query(Nav.filter())
async def navigate(query: CallbackQuery, callback_data: Nav, state: FSMContext) -> None:
+111
View File
@@ -0,0 +1,111 @@
"""«👥 Участники» for admins: list, invite by QR, revoke access (SPEC A-1, A-7)."""
from datetime import datetime
from enum import StrEnum
from aiogram import Bot, F, Router
from aiogram.filters.callback_data import CallbackData
from aiogram.fsm.context import FSMContext
from aiogram.types import BufferedInputFile, CallbackQuery, InlineKeyboardMarkup, Message
from aiogram.utils.keyboard import InlineKeyboardBuilder
from app.access.qr import invite_link, qr_png
from app.access.service import AccessService
from app.bot.menu import Nav, NavAction
from app.db.models import Role, User
from app.texts import Texts
class MembersAction(StrEnum):
INVITE = "inv"
ASK_REVOKE = "ask"
REVOKE = "rev"
LIST = "list"
class MembersCb(CallbackData, prefix="mem"):
action: MembersAction
tg_id: int = 0
def _fmt(dt: datetime | None, never: str) -> str:
return dt.strftime("%d.%m %H:%M") if dt else never
def render_members(texts: Texts, members: list[User]) -> tuple[str, InlineKeyboardMarkup]:
t = texts.members
lines = [t.title.format(count=len(members)), ""]
for m in members:
lines.append(
t.line.format(
icon="👑" if m.role is Role.ADMIN else "👤",
name=m.full_name,
username=f" @{m.username}" if m.username else "",
joined=_fmt(m.joined_at, t.never),
seen=_fmt(m.last_seen_at, t.never),
)
)
kb = InlineKeyboardBuilder()
kb.button(text=t.invite, callback_data=MembersCb(action=MembersAction.INVITE))
for m in members:
if m.role is not Role.ADMIN:
kb.button(
text=t.revoke.format(name=m.full_name),
callback_data=MembersCb(action=MembersAction.ASK_REVOKE, tg_id=m.tg_id),
)
kb.button(text=texts.menu.back, callback_data=Nav(action=NavAction.BACK))
kb.adjust(1)
return "\n".join(lines), kb.as_markup()
def build_router(texts: Texts, access: AccessService) -> Router:
router = Router(name="members")
t = texts.members
# Every handler here is admin-only; a member who somehow has the button (e.g. was demoted) gets nothing.
def is_admin(member: User | None) -> bool:
return member is not None and member.role is Role.ADMIN
@router.message(F.text == texts.menu.members)
async def open_members(message: Message, state: FSMContext, member: User) -> None:
if not is_admin(member):
return
await state.clear()
text, kb = render_members(texts, await access.list_members())
await message.answer(text, reply_markup=kb)
@router.callback_query(MembersCb.filter())
async def on_action(query: CallbackQuery, callback_data: MembersCb, bot: Bot, member: User) -> None:
if not is_admin(member) or not isinstance(query.message, Message):
await query.answer()
return
msg = query.message
match callback_data.action:
case MembersAction.INVITE:
invite = await access.create_invite(member.tg_id)
me = await bot.me()
link = invite_link(me.username or "", invite.token)
caption = t.invite_caption.format(link=link, expires=invite.expires_at.strftime("%d.%m %H:%M"))
await msg.answer_photo(BufferedInputFile(qr_png(link), "invite.png"), caption=caption)
case MembersAction.ASK_REVOKE:
target = await access.get_member(callback_data.tg_id)
name = target.full_name if target else str(callback_data.tg_id)
kb = InlineKeyboardBuilder()
kb.button(
text=t.confirm_yes, callback_data=MembersCb(action=MembersAction.REVOKE, tg_id=callback_data.tg_id)
)
kb.button(text=t.confirm_no, callback_data=MembersCb(action=MembersAction.LIST))
await msg.edit_text(t.confirm.format(name=name), reply_markup=kb.as_markup())
case MembersAction.REVOKE:
target = await access.get_member(callback_data.tg_id)
name = target.full_name if target else str(callback_data.tg_id)
ok = await access.revoke(callback_data.tg_id)
text, kb_markup = render_members(texts, await access.list_members())
note = t.revoked.format(name=name) if ok else t.not_revoked
await msg.edit_text(f"{note}\n\n{text}", reply_markup=kb_markup)
case MembersAction.LIST:
text, kb_markup = render_members(texts, await access.list_members())
await msg.edit_text(text, reply_markup=kb_markup)
await query.answer()
return router
+216
View File
@@ -0,0 +1,216 @@
"""«🩺 Статус»: managed containers, a card per container, on-demand checks (SPEC S-1…S-4)."""
import logging
from datetime import datetime
from enum import StrEnum
from aiogram import F, Router
from aiogram.exceptions import TelegramBadRequest
from aiogram.filters.callback_data import CallbackData
from aiogram.fsm.context import FSMContext
from aiogram.types import CallbackQuery, InlineKeyboardMarkup, Message
from aiogram.utils.keyboard import InlineKeyboardBuilder
from app.bot.menu import Nav, NavAction
from app.bot.routers.commit import human_size
from app.db.models import utcnow
from app.runtime.checks import Checker, CheckName, ContainerReport
from app.runtime.docker import Container, Details, DockerClient, DockerError, Stats
from app.texts import StatusTexts, Texts
log = logging.getLogger(__name__)
MAX_TEXT = 4000 # Telegram's limit is 4096
class StatusAct(StrEnum):
LIST = "l"
CARD = "c"
CHECK = "k"
class StatusCb(CallbackData, prefix="s"):
a: StatusAct
id: str = "" # 12-char container id prefix: callback data is limited to 64 bytes
def duration(t: StatusTexts, seconds: float) -> str:
s = int(max(seconds, 0))
d, s = divmod(s, 86400)
h, s = divmod(s, 3600)
m, s = divmod(s, 60)
if d:
return f"{d} {t.days} {h} {t.hours}"
if h:
return f"{h} {t.hours} {m} {t.minutes}"
if m:
return f"{m} {t.minutes}"
return f"{s} {t.seconds}"
def _clip(text: str) -> str:
return text if len(text) <= MAX_TEXT else text[: MAX_TEXT - 1] + "…"
def _fmt_time(dt: datetime | None, none: str) -> str:
return dt.strftime("%d.%m %H:%M") if dt else none
def render_list(texts: Texts, containers: list[Container], now: datetime) -> tuple[str, InlineKeyboardMarkup]:
t = texts.status
lines = [t.title.format(count=len(containers), time=now.strftime("%H:%M:%S")), ""]
if not containers:
lines.append(t.empty)
for c in containers:
state = c.state if c.health is None else f"{c.state}, {c.health}"
lines.append(t.line.format(light=c.light.value, name=c.name, state=state))
kb = InlineKeyboardBuilder()
for c in containers:
kb.button(text=f"{c.light.value} {c.name}", callback_data=StatusCb(a=StatusAct.CARD, id=c.short_id))
if containers:
kb.button(text=t.check_button, callback_data=StatusCb(a=StatusAct.CHECK))
kb.button(text=t.refresh, callback_data=StatusCb(a=StatusAct.LIST))
kb.button(text=texts.menu.back, callback_data=Nav(action=NavAction.BACK))
kb.adjust(1)
return _clip("\n".join(lines)), kb.as_markup()
def render_card(texts: Texts, d: Details, stats: Stats | None, now: datetime) -> tuple[str, InlineKeyboardMarkup]:
t = texts.status
running = d.state == "running"
uptime = duration(t, (now - d.started_at).total_seconds()) if running and d.started_at else t.none
if stats is None:
cpu = mem = limit = mem_pct = t.none
else:
cpu = f"{stats.cpu_pct:.1f}%"
mem = human_size(stats.mem_bytes)
limit = human_size(stats.mem_limit) if stats.mem_limit else t.no_limit
mem_pct = f"{stats.mem_pct:.0f}%"
text = t.card.format(
light=d.light.value,
name=d.name,
service=d.service or t.none,
image=d.image or t.none,
state=d.state if running else f"{d.state} (exit {d.exit_code}{', OOM' if d.oom_killed else ''})",
health=d.health or t.none,
uptime=uptime,
restarts=d.restart_count,
cpu=cpu,
mem=mem,
limit=limit,
mem_pct=mem_pct,
sha=(d.sha or t.none)[:12],
deployed=_fmt_time(d.deployed_at, t.none),
)
kb = InlineKeyboardBuilder()
kb.button(text=t.check_button, callback_data=StatusCb(a=StatusAct.CHECK, id=d.id[:12]))
kb.button(text=t.refresh, callback_data=StatusCb(a=StatusAct.CARD, id=d.id[:12]))
kb.button(text=t.to_list, callback_data=StatusCb(a=StatusAct.LIST))
kb.adjust(2, 1)
return text, kb.as_markup()
def render_report(
texts: Texts, reports: list[ContainerReport], now: datetime, back_id: str = ""
) -> tuple[str, InlineKeyboardMarkup]:
"""back_id: the report was run from a card — offer the way back to it."""
t = texts.status
titles = {
CheckName.RUNNING: t.check_running,
CheckName.HEALTH: t.check_health,
CheckName.RESTARTS: t.check_restarts,
CheckName.CPU: t.check_cpu,
CheckName.MEM: t.check_mem,
CheckName.LOG_ERRORS: t.check_log_errors,
CheckName.HTTP: t.check_http,
}
ok = sum(r.ok for r in reports)
lines = [t.report_title.format(ok=ok, total=len(reports), time=now.strftime("%H:%M:%S"))]
for report in reports:
lines.append("")
lines.append(t.report_container.format(light="✅" if report.ok else "❌", name=report.details.name))
for r in report.results:
detail = f" · {r.detail}" if r.detail else ""
lines.append(t.report_line.format(verdict=r.verdict.value, title=titles[r.name], detail=detail))
kb = InlineKeyboardBuilder()
kb.button(text=t.refresh, callback_data=StatusCb(a=StatusAct.CHECK, id=back_id))
if back_id:
kb.button(text=t.to_card, callback_data=StatusCb(a=StatusAct.CARD, id=back_id))
kb.button(text=t.to_list, callback_data=StatusCb(a=StatusAct.LIST))
kb.adjust(1)
return _clip("\n".join(lines)), kb.as_markup()
def build_router(texts: Texts, docker: DockerClient | None, checker: Checker | None) -> Router:
router = Router(name="status")
t = texts.status
async def list_screen() -> tuple[str, InlineKeyboardMarkup]:
if docker is None:
raise DockerError("docker is not configured (DOCKER_URL)")
return render_list(texts, await docker.containers(), utcnow())
async def card_screen(short_id: str) -> tuple[str, InlineKeyboardMarkup]:
if docker is None:
raise DockerError("docker is not configured (DOCKER_URL)")
cid = await docker.resolve(short_id)
details = await docker.inspect(cid)
stats = None
if details.state == "running":
try:
stats = await docker.stats(cid)
except DockerError:
log.warning("stats %s unavailable", short_id)
return render_card(texts, details, stats, utcnow())
@router.message(F.text == texts.menu.status)
async def open_status(message: Message, state: FSMContext) -> None:
await state.clear()
try:
text, kb = await list_screen()
except DockerError as e:
log.warning("status: %s", e)
await message.answer(t.unavailable)
return
await message.answer(text, reply_markup=kb)
@router.callback_query(StatusCb.filter())
async def on_action(query: CallbackQuery, callback_data: StatusCb) -> None:
if not isinstance(query.message, Message):
await query.answer()
return
msg = query.message
try:
match callback_data.a:
case StatusAct.LIST:
text, kb = await list_screen()
case StatusAct.CARD:
text, kb = await card_screen(callback_data.id)
case StatusAct.CHECK:
if docker is None or checker is None:
raise DockerError("docker is not configured (DOCKER_URL)")
await query.answer(t.checking)
if callback_data.id:
reports = [await checker.check(await docker.resolve(callback_data.id))]
else:
reports = await checker.check_all()
text, kb = render_report(texts, reports, utcnow(), callback_data.id)
await _edit(msg, text, kb)
return
except DockerError as e:
log.warning("status: %s", e)
await query.answer(t.unavailable, show_alert=True)
return
await _edit(msg, text, kb)
await query.answer()
return router
async def _edit(msg: Message, text: str, kb: InlineKeyboardMarkup) -> None:
# «🔄 Обновить» with nothing changed: Telegram rejects an identical edit — not an error for the user
try:
await msg.edit_text(text, reply_markup=kb)
except TelegramBadRequest as e:
if "message is not modified" not in str(e):
raise
+58
View File
@@ -13,6 +13,8 @@ class TelegramSettings(BaseSettings):
model_config = SettingsConfigDict(**_ENV, env_prefix="TELEGRAM_")
bot_token: SecretStr
# http://user:pass@host:port or socks5://… — the RF server has no route to api.telegram.org (same as meditron)
proxy_url: SecretStr | None = None
class AccessSettings(BaseSettings):
@@ -20,6 +22,9 @@ class AccessSettings(BaseSettings):
# Bootstrap admins (SPEC A-8). Comma-separated in the env: ADMIN_IDS=1,2
admin_ids: Annotated[frozenset[int], NoDecode] = frozenset()
invite_ttl_hours: float = Field(default=24.0, gt=0)
# More failed /start <token> per hour than this — silence (SPEC A-6)
max_failed_attempts: int = Field(default=5, ge=1)
@field_validator("admin_ids", mode="before")
@classmethod
@@ -42,6 +47,55 @@ class LLMSettings(BaseSettings):
temperature: float = Field(default=0.0, ge=0, le=2)
class DatabaseSettings(BaseSettings):
model_config = SettingsConfigDict(**_ENV, env_prefix="DATABASE_")
# Compose sets a PostgreSQL URL; the default lets the bot run locally without Docker.
url: str = "sqlite+aiosqlite:///./tgit.db"
class SourceCraftSettings(BaseSettings):
model_config = SettingsConfigDict(**_ENV, env_prefix="SOURCECRAFT_")
token: SecretStr = SecretStr("")
api_url: str = "https://api.sourcecraft.tech"
uploads_repo: str = "rogachevegorushka/tgit-uploads"
# Web link to a CI run. The SPA answers 200 on any path, so the format is unverified — adjust if it 404s in UI.
run_url_template: str = "https://sourcecraft.dev/{repo}/cicd/runs/{run}"
timeout: float = Field(default=30.0, gt=0)
max_attempts: int = Field(default=5, ge=1)
class GitSettings(BaseSettings):
model_config = SettingsConfigDict(**_ENV, env_prefix="GIT_")
# The bot's own deploy key with write access to tgit-uploads only (#7)
uploads_url: str = "git@ssh.sourcecraft.dev:rogachevegorushka/tgit-uploads.git"
ssh_key_path: Path | None = None
# Alternative for containers: the private key itself, base64 — a mounted host file is unreadable for the
# container user (uid 10001 vs the owner's 0600). Written to a private temp file at startup.
ssh_key_b64: SecretStr | None = None
branch: str = "main"
# Web link to a commit in the uploads repo, e.g. https://sourcecraft.dev/{repo}/commit/{sha}; empty — no link
commit_url_template: str = ""
clone_dir: Path = Path("data/uploads")
# Browsing re-fetches at most this often; a push always fetches first (#9)
fetch_interval: float = Field(default=10.0, ge=0)
timeout: float = Field(default=60.0, gt=0)
class DockerSettings(BaseSettings):
model_config = SettingsConfigDict(**_ENV, env_prefix="DOCKER_")
# docker-socket-proxy (GET only, ADR-006); empty — the «Статус» section reports Docker as unavailable
url: str = ""
# Only containers with this label are shown: the host runs other projects too
label: str = "tgit.managed=true"
timeout: float = Field(default=10.0, gt=0)
snapshot_interval: float = Field(default=15.0, gt=0)
max_log_bytes: int = Field(default=5 * 1024 * 1024, gt=0)
class RuntimeSettings(BaseSettings):
model_config = _ENV
@@ -59,5 +113,9 @@ class Settings(BaseSettings):
telegram: TelegramSettings = Field(default_factory=TelegramSettings)
access: AccessSettings = Field(default_factory=AccessSettings)
database: DatabaseSettings = Field(default_factory=DatabaseSettings)
llm: LLMSettings = Field(default_factory=LLMSettings)
sourcecraft: SourceCraftSettings = Field(default_factory=SourceCraftSettings)
git: GitSettings = Field(default_factory=GitSettings)
docker: DockerSettings = Field(default_factory=DockerSettings)
runtime: RuntimeSettings = Field(default_factory=RuntimeSettings)
View File
+46
View File
@@ -0,0 +1,46 @@
"""Periodic cleanup of TTL data: expired drafts, old container snapshots and access attempts."""
import asyncio
import logging
from datetime import datetime, timedelta
from typing import Any, cast
from sqlalchemy import CursorResult, Delete, delete
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import AccessAttempt, ContainerSnapshot, Draft, utcnow
from app.db.session import SessionMaker
log = logging.getLogger(__name__)
SNAPSHOT_TTL = timedelta(days=1)
ATTEMPT_TTL = timedelta(days=1)
async def _delete(session: AsyncSession, stmt: Delete) -> int:
# DML executes into a CursorResult; the generic annotation of execute() just doesn't say so
return cast(CursorResult[Any], await session.execute(stmt)).rowcount
async def cleanup_expired(sessions: SessionMaker, now: datetime) -> dict[str, int]:
async with sessions.begin() as session:
return {
"drafts": await _delete(session, delete(Draft).where(Draft.expires_at < now)),
"snapshots": await _delete(
session, delete(ContainerSnapshot).where(ContainerSnapshot.at < now - SNAPSHOT_TTL)
),
"access_attempts": await _delete(
session, delete(AccessAttempt).where(AccessAttempt.at < now - ATTEMPT_TTL)
),
}
async def cleanup_loop(sessions: SessionMaker, interval: float = 3600.0) -> None:
while True:
try:
removed = await cleanup_expired(sessions, utcnow())
if any(removed.values()):
log.info("cleanup: %s", removed)
except Exception: # a failed cleanup must not kill the bot; retry next round
log.exception("cleanup failed")
await asyncio.sleep(interval)
+52
View File
@@ -0,0 +1,52 @@
"""Apply Alembic migrations programmatically at startup — no separate step in the container.
`python -m app.db.migrate [revision]` upgrades the DB from DATABASE_URL (default: head).
"""
import asyncio
import sys
from alembic import command
from alembic.config import Config
from sqlalchemy.engine import Connection
from sqlalchemy.ext.asyncio import AsyncEngine
from app.config import DatabaseSettings
from app.db.session import make_engine
SCRIPT_LOCATION = "app.db:migrations"
def alembic_config(connection: Connection | None = None) -> Config:
cfg = Config()
cfg.set_main_option("script_location", SCRIPT_LOCATION)
# env.py reuses this connection instead of opening its own (async engine, one transaction)
cfg.attributes["connection"] = connection
return cfg
async def upgrade(engine: AsyncEngine, revision: str = "head") -> None:
async with engine.begin() as conn:
await conn.run_sync(lambda sync_conn: command.upgrade(alembic_config(sync_conn), revision))
async def downgrade(engine: AsyncEngine, revision: str) -> None:
async with engine.begin() as conn:
await conn.run_sync(lambda sync_conn: command.downgrade(alembic_config(sync_conn), revision))
def main() -> None:
revision = sys.argv[1] if len(sys.argv) > 1 else "head"
async def _run() -> None:
engine = make_engine(DatabaseSettings().url)
try:
await upgrade(engine, revision)
finally:
await engine.dispose()
asyncio.run(_run())
if __name__ == "__main__":
main()
View File
+39
View File
@@ -0,0 +1,39 @@
"""Alembic environment. Runs on a connection handed over by app.db.migrate (or the CLI via alembic.ini)."""
import asyncio
from alembic import context
from sqlalchemy.engine import Connection
from app.config import DatabaseSettings
from app.db.models import Base
from app.db.session import make_engine
target_metadata = Base.metadata
def _run(connection: Connection) -> None:
context.configure(
connection=connection,
target_metadata=target_metadata,
render_as_batch=connection.dialect.name == "sqlite", # SQLite can't ALTER most things in place
compare_type=True,
)
with context.begin_transaction():
context.run_migrations()
async def _run_cli() -> None:
engine = make_engine(DatabaseSettings().url)
try:
async with engine.begin() as conn:
await conn.run_sync(_run)
finally:
await engine.dispose()
connection = context.config.attributes.get("connection")
if connection is not None:
_run(connection)
else:
asyncio.run(_run_cli())
+25
View File
@@ -0,0 +1,25 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
${imports if imports else ""}
revision: str = ${repr(up_revision)}
down_revision: str | None = ${repr(down_revision)}
branch_labels: str | Sequence[str] | None = ${repr(branch_labels)}
depends_on: str | Sequence[str] | None = ${repr(depends_on)}
def upgrade() -> None:
${upgrades if upgrades else "pass"}
def downgrade() -> None:
${downgrades if downgrades else "pass"}
@@ -0,0 +1,311 @@
"""initial schema
Revision ID: 0001
Revises:
Create Date: 2026-10-14 17:41:05.672912
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "0001"
down_revision: str | None = None
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.create_table(
"access_attempts",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("tg_id", sa.BigInteger(), nullable=False),
sa.Column("at", sa.DateTime(timezone=True), nullable=False),
sa.Column("ok", sa.Boolean(), nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("pk_access_attempts")),
)
with op.batch_alter_table("access_attempts", schema=None) as batch_op:
batch_op.create_index("ix_access_attempts_tg_id_at", ["tg_id", "at"], unique=False)
op.create_table(
"alerts",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column(
"kind",
sa.Enum(
"crash",
"oom",
"restart_loop",
"unhealthy",
"overload",
"deploy_refused",
"cd_failed",
"disk_low",
"dependency_down",
"agent_attack",
name="alert_kind",
native_enum=False,
create_constraint=True,
length=20,
),
nullable=False,
),
sa.Column("service", sa.String(length=100), nullable=True),
sa.Column(
"payload", sa.JSON().with_variant(postgresql.JSONB(astext_type=sa.Text()), "postgresql"), nullable=False
),
sa.Column("count", sa.Integer(), nullable=False),
sa.Column("opened_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("last_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("resolved_at", sa.DateTime(timezone=True), nullable=True),
sa.PrimaryKeyConstraint("id", name=op.f("pk_alerts")),
)
with op.batch_alter_table("alerts", schema=None) as batch_op:
batch_op.create_index("ix_alerts_kind_service_resolved_at", ["kind", "service", "resolved_at"], unique=False)
op.create_table(
"container_snapshots",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("container", sa.String(length=100), nullable=False),
sa.Column("at", sa.DateTime(timezone=True), nullable=False),
sa.Column("restart_count", sa.Integer(), nullable=False),
sa.Column("cpu_pct", sa.Float(), nullable=False),
sa.Column("mem_pct", sa.Float(), nullable=False),
sa.Column("health", sa.String(length=20), nullable=True),
sa.PrimaryKeyConstraint("id", name=op.f("pk_container_snapshots")),
)
with op.batch_alter_table("container_snapshots", schema=None) as batch_op:
batch_op.create_index("ix_container_snapshots_container_at", ["container", "at"], unique=False)
op.create_table(
"fsm_state",
sa.Column("key", sa.String(length=255), nullable=False),
sa.Column("state", sa.String(length=255), nullable=True),
sa.Column(
"data", sa.JSON().with_variant(postgresql.JSONB(astext_type=sa.Text()), "postgresql"), nullable=False
),
sa.PrimaryKeyConstraint("key", name=op.f("pk_fsm_state")),
)
op.create_table(
"users",
sa.Column("tg_id", sa.BigInteger(), autoincrement=False, nullable=False),
sa.Column("username", sa.String(length=64), nullable=True),
sa.Column("full_name", sa.String(length=256), nullable=False),
sa.Column(
"role",
sa.Enum("member", "admin", name="role", native_enum=False, create_constraint=True, length=20),
nullable=False,
),
sa.Column("is_active", sa.Boolean(), nullable=False),
sa.Column("invited_by", sa.BigInteger(), nullable=True),
sa.Column("joined_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("last_seen_at", sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(["invited_by"], ["users.tg_id"], name=op.f("fk_users_invited_by_users")),
sa.PrimaryKeyConstraint("tg_id", name=op.f("pk_users")),
)
op.create_table(
"agent_requests",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("user_id", sa.BigInteger(), nullable=False),
sa.Column("prompt", sa.Text(), nullable=False),
sa.Column(
"verdict",
sa.Enum("safe", "attack", "error", name="verdict", native_enum=False, create_constraint=True, length=20),
nullable=False,
),
sa.Column("reason", sa.Text(), nullable=True),
sa.Column(
"log_query", sa.JSON().with_variant(postgresql.JSONB(astext_type=sa.Text()), "postgresql"), nullable=True
),
sa.Column("tokens", sa.Integer(), nullable=False),
sa.Column("at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["user_id"], ["users.tg_id"], name=op.f("fk_agent_requests_user_id_users")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_agent_requests")),
)
op.create_table(
"alert_deliveries",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("alert_id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("chat_id", sa.BigInteger(), nullable=False),
sa.Column("message_id", sa.BigInteger(), nullable=False),
sa.ForeignKeyConstraint(
["alert_id"], ["alerts.id"], name=op.f("fk_alert_deliveries_alert_id_alerts"), ondelete="CASCADE"
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_alert_deliveries")),
sa.UniqueConstraint("alert_id", "chat_id", name=op.f("uq_alert_deliveries_alert_id")),
)
op.create_table(
"drafts",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("user_id", sa.BigInteger(), nullable=False),
sa.Column("base_sha", sa.String(length=40), nullable=False),
sa.Column("message", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["user_id"], ["users.tg_id"], name=op.f("fk_drafts_user_id_users"), ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id", name=op.f("pk_drafts")),
sa.UniqueConstraint("user_id", name=op.f("uq_drafts_user_id")),
)
op.create_table(
"invites",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("token_hash", sa.String(length=64), nullable=False),
sa.Column("created_by", sa.BigInteger(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("used_by", sa.BigInteger(), nullable=True),
sa.Column("used_at", sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(["created_by"], ["users.tg_id"], name=op.f("fk_invites_created_by_users")),
sa.ForeignKeyConstraint(["used_by"], ["users.tg_id"], name=op.f("fk_invites_used_by_users")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_invites")),
sa.UniqueConstraint("token_hash", name=op.f("uq_invites_token_hash")),
)
op.create_table(
"log_queries",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("user_id", sa.BigInteger(), nullable=False),
sa.Column("container", sa.String(length=100), nullable=False),
sa.Column(
"mode",
sa.Enum("tail", "grep", "agent", name="log_mode", native_enum=False, create_constraint=True, length=20),
nullable=False,
),
sa.Column(
"query", sa.JSON().with_variant(postgresql.JSONB(astext_type=sa.Text()), "postgresql"), nullable=False
),
sa.Column("lines", sa.Integer(), nullable=False),
sa.Column("at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["user_id"], ["users.tg_id"], name=op.f("fk_log_queries_user_id_users")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_log_queries")),
)
op.create_table(
"pushes",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("user_id", sa.BigInteger(), nullable=False),
sa.Column("sha", sa.String(length=40), nullable=False),
sa.Column("message", sa.Text(), nullable=False),
sa.Column(
"paths", sa.JSON().with_variant(postgresql.JSONB(astext_type=sa.Text()), "postgresql"), nullable=False
),
sa.Column("chat_id", sa.BigInteger(), nullable=False),
sa.Column("message_id", sa.BigInteger(), nullable=True),
sa.Column(
"ci_status",
sa.Enum(
"pending",
"running",
"success",
"failed",
"timeout",
"skipped",
name="ci_status",
native_enum=False,
create_constraint=True,
length=20,
),
nullable=False,
),
sa.Column(
"cd_status",
sa.Enum(
"pending",
"running",
"success",
"failed",
"timeout",
"skipped",
name="cd_status",
native_enum=False,
create_constraint=True,
length=20,
),
nullable=False,
),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["user_id"], ["users.tg_id"], name=op.f("fk_pushes_user_id_users")),
sa.PrimaryKeyConstraint("id", name=op.f("pk_pushes")),
sa.UniqueConstraint("sha", name=op.f("uq_pushes_sha")),
)
op.create_table(
"draft_changes",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("draft_id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("path", sa.String(length=500), nullable=False),
sa.Column(
"op",
sa.Enum("add", "modify", "delete", name="change_op", native_enum=False, create_constraint=True, length=20),
nullable=False,
),
sa.Column("content", sa.LargeBinary(), nullable=True),
sa.ForeignKeyConstraint(
["draft_id"], ["drafts.id"], name=op.f("fk_draft_changes_draft_id_drafts"), ondelete="CASCADE"
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_draft_changes")),
sa.UniqueConstraint("draft_id", "path", name=op.f("uq_draft_changes_draft_id")),
)
op.create_table(
"pipeline_runs",
sa.Column("id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column("push_id", sa.BigInteger().with_variant(sa.Integer(), "sqlite"), nullable=False),
sa.Column(
"stage",
sa.Enum("ci", "cd", name="stage", native_enum=False, create_constraint=True, length=20),
nullable=False,
),
sa.Column("run_slug", sa.String(length=50), nullable=False),
sa.Column(
"status",
sa.Enum(
"pending",
"running",
"success",
"failed",
"timeout",
"skipped",
name="run_status",
native_enum=False,
create_constraint=True,
length=20,
),
nullable=False,
),
sa.Column("failed_cube", sa.String(length=100), nullable=True),
sa.Column("url", sa.String(length=500), nullable=True),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["push_id"], ["pushes.id"], name=op.f("fk_pipeline_runs_push_id_pushes"), ondelete="CASCADE"
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_pipeline_runs")),
sa.UniqueConstraint("push_id", "stage", name=op.f("uq_pipeline_runs_push_id")),
)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.drop_table("pipeline_runs")
op.drop_table("draft_changes")
op.drop_table("pushes")
op.drop_table("log_queries")
op.drop_table("invites")
op.drop_table("drafts")
op.drop_table("alert_deliveries")
op.drop_table("agent_requests")
op.drop_table("users")
op.drop_table("fsm_state")
with op.batch_alter_table("container_snapshots", schema=None) as batch_op:
batch_op.drop_index("ix_container_snapshots_container_at")
op.drop_table("container_snapshots")
with op.batch_alter_table("alerts", schema=None) as batch_op:
batch_op.drop_index("ix_alerts_kind_service_resolved_at")
op.drop_table("alerts")
with op.batch_alter_table("access_attempts", schema=None) as batch_op:
batch_op.drop_index("ix_access_attempts_tg_id_at")
op.drop_table("access_attempts")
# ### end Alembic commands ###
+283
View File
@@ -0,0 +1,283 @@
"""Data model (ARCHITECTURE §2.10). One schema for PostgreSQL (prod) and SQLite (tests).
Enums are stored as VARCHAR + CHECK rather than native PG enums: portable and painless to migrate.
"""
from datetime import UTC, datetime
from enum import StrEnum
from typing import Any, ClassVar
from sqlalchemy import (
JSON,
BigInteger,
Boolean,
DateTime,
Enum,
Float,
ForeignKey,
Index,
Integer,
LargeBinary,
MetaData,
String,
Text,
UniqueConstraint,
)
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship
# Stable constraint names so Alembic migrations stay deterministic across databases.
NAMING = {
"ix": "ix_%(column_0_label)s",
"uq": "uq_%(table_name)s_%(column_0_name)s",
"ck": "ck_%(table_name)s_%(constraint_name)s",
"fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
"pk": "pk_%(table_name)s",
}
# Surrogate keys: BIGINT on Postgres, INTEGER on SQLite (only INTEGER PRIMARY KEY autoincrements there).
Id = BigInteger().with_variant(Integer(), "sqlite")
Json = JSON().with_variant(JSONB(), "postgresql")
TgId = BigInteger
def utcnow() -> datetime:
return datetime.now(UTC)
def enum(cls: type[StrEnum], name: str) -> Enum:
return Enum(
cls,
name=name,
native_enum=False,
create_constraint=True, # SQLAlchemy 2 skips the CHECK for non-native enums unless asked
length=20,
values_callable=lambda e: [m.value for m in e],
)
class Role(StrEnum):
MEMBER = "member"
ADMIN = "admin"
class ChangeOp(StrEnum):
ADD = "add"
MODIFY = "modify"
DELETE = "delete"
class Stage(StrEnum):
CI = "ci"
CD = "cd"
class PipelineStatus(StrEnum):
PENDING = "pending"
RUNNING = "running"
SUCCESS = "success"
FAILED = "failed"
TIMEOUT = "timeout"
SKIPPED = "skipped"
class AlertKind(StrEnum):
CRASH = "crash"
OOM = "oom"
RESTART_LOOP = "restart_loop"
UNHEALTHY = "unhealthy"
OVERLOAD = "overload"
DEPLOY_REFUSED = "deploy_refused"
CD_FAILED = "cd_failed"
DISK_LOW = "disk_low"
DEPENDENCY_DOWN = "dependency_down"
AGENT_ATTACK = "agent_attack"
class LogMode(StrEnum):
TAIL = "tail"
GREP = "grep"
AGENT = "agent"
class Verdict(StrEnum):
SAFE = "safe"
ATTACK = "attack"
ERROR = "error"
class Base(DeclarativeBase):
metadata = MetaData(naming_convention=NAMING)
type_annotation_map: ClassVar[dict[Any, Any]] = {
datetime: DateTime(timezone=True),
dict[str, Any]: Json,
list[str]: Json,
}
class User(Base):
__tablename__ = "users"
tg_id: Mapped[int] = mapped_column(TgId, primary_key=True, autoincrement=False)
username: Mapped[str | None] = mapped_column(String(64))
full_name: Mapped[str] = mapped_column(String(256))
role: Mapped[Role] = mapped_column(enum(Role, "role"), default=Role.MEMBER)
is_active: Mapped[bool] = mapped_column(Boolean, default=True)
invited_by: Mapped[int | None] = mapped_column(TgId, ForeignKey("users.tg_id"))
joined_at: Mapped[datetime] = mapped_column(default=utcnow)
last_seen_at: Mapped[datetime | None]
class Invite(Base):
__tablename__ = "invites"
id: Mapped[int] = mapped_column(Id, primary_key=True)
# sha256 hex of the token; the token itself is never stored (SPEC A-3)
token_hash: Mapped[str] = mapped_column(String(64), unique=True)
created_by: Mapped[int] = mapped_column(TgId, ForeignKey("users.tg_id"))
created_at: Mapped[datetime] = mapped_column(default=utcnow)
expires_at: Mapped[datetime]
used_by: Mapped[int | None] = mapped_column(TgId, ForeignKey("users.tg_id"))
used_at: Mapped[datetime | None]
class AccessAttempt(Base):
__tablename__ = "access_attempts"
__table_args__ = (Index("ix_access_attempts_tg_id_at", "tg_id", "at"),)
id: Mapped[int] = mapped_column(Id, primary_key=True)
tg_id: Mapped[int] = mapped_column(TgId)
at: Mapped[datetime] = mapped_column(default=utcnow)
ok: Mapped[bool] = mapped_column(Boolean)
class FsmState(Base):
__tablename__ = "fsm_state"
key: Mapped[str] = mapped_column(String(255), primary_key=True)
state: Mapped[str | None] = mapped_column(String(255))
data: Mapped[dict[str, Any]] = mapped_column(default=dict)
class Draft(Base):
__tablename__ = "drafts"
id: Mapped[int] = mapped_column(Id, primary_key=True)
user_id: Mapped[int] = mapped_column(TgId, ForeignKey("users.tg_id", ondelete="CASCADE"), unique=True)
base_sha: Mapped[str] = mapped_column(String(40))
message: Mapped[str | None] = mapped_column(Text)
created_at: Mapped[datetime] = mapped_column(default=utcnow)
expires_at: Mapped[datetime]
changes: Mapped[list["DraftChange"]] = relationship(
back_populates="draft", cascade="all, delete-orphan", order_by="DraftChange.id"
)
class DraftChange(Base):
__tablename__ = "draft_changes"
__table_args__ = (UniqueConstraint("draft_id", "path"),)
id: Mapped[int] = mapped_column(Id, primary_key=True)
draft_id: Mapped[int] = mapped_column(Id, ForeignKey("drafts.id", ondelete="CASCADE"))
path: Mapped[str] = mapped_column(String(500))
op: Mapped[ChangeOp] = mapped_column(enum(ChangeOp, "change_op"))
content: Mapped[bytes | None] = mapped_column(LargeBinary)
draft: Mapped[Draft] = relationship(back_populates="changes")
class Push(Base):
__tablename__ = "pushes"
id: Mapped[int] = mapped_column(Id, primary_key=True)
user_id: Mapped[int] = mapped_column(TgId, ForeignKey("users.tg_id"))
sha: Mapped[str] = mapped_column(String(40), unique=True)
message: Mapped[str] = mapped_column(Text)
paths: Mapped[list[str]] = mapped_column(default=list)
chat_id: Mapped[int] = mapped_column(TgId)
message_id: Mapped[int | None] = mapped_column(BigInteger)
ci_status: Mapped[PipelineStatus] = mapped_column(enum(PipelineStatus, "ci_status"), default=PipelineStatus.PENDING)
cd_status: Mapped[PipelineStatus] = mapped_column(enum(PipelineStatus, "cd_status"), default=PipelineStatus.PENDING)
created_at: Mapped[datetime] = mapped_column(default=utcnow)
class PipelineRun(Base):
__tablename__ = "pipeline_runs"
__table_args__ = (UniqueConstraint("push_id", "stage"),)
id: Mapped[int] = mapped_column(Id, primary_key=True)
push_id: Mapped[int] = mapped_column(Id, ForeignKey("pushes.id", ondelete="CASCADE"))
# CI and CD are tasks of one SourceCraft workflow (check-and-deploy), hence "stage", not "workflow"
stage: Mapped[Stage] = mapped_column(enum(Stage, "stage"))
run_slug: Mapped[str] = mapped_column(String(50))
status: Mapped[PipelineStatus] = mapped_column(enum(PipelineStatus, "run_status"))
failed_cube: Mapped[str | None] = mapped_column(String(100))
url: Mapped[str | None] = mapped_column(String(500))
updated_at: Mapped[datetime] = mapped_column(default=utcnow, onupdate=utcnow)
class ContainerSnapshot(Base):
__tablename__ = "container_snapshots"
__table_args__ = (Index("ix_container_snapshots_container_at", "container", "at"),)
id: Mapped[int] = mapped_column(Id, primary_key=True)
container: Mapped[str] = mapped_column(String(100))
at: Mapped[datetime] = mapped_column(default=utcnow)
restart_count: Mapped[int] = mapped_column(Integer)
cpu_pct: Mapped[float] = mapped_column(Float)
mem_pct: Mapped[float] = mapped_column(Float)
health: Mapped[str | None] = mapped_column(String(20))
class Alert(Base):
__tablename__ = "alerts"
__table_args__ = (Index("ix_alerts_kind_service_resolved_at", "kind", "service", "resolved_at"),)
id: Mapped[int] = mapped_column(Id, primary_key=True)
kind: Mapped[AlertKind] = mapped_column(enum(AlertKind, "alert_kind"))
service: Mapped[str | None] = mapped_column(String(100))
payload: Mapped[dict[str, Any]] = mapped_column(default=dict)
count: Mapped[int] = mapped_column(Integer, default=1)
opened_at: Mapped[datetime] = mapped_column(default=utcnow)
last_at: Mapped[datetime] = mapped_column(default=utcnow)
resolved_at: Mapped[datetime | None]
deliveries: Mapped[list["AlertDelivery"]] = relationship(back_populates="alert", cascade="all, delete-orphan")
class AlertDelivery(Base):
__tablename__ = "alert_deliveries"
__table_args__ = (UniqueConstraint("alert_id", "chat_id"),)
id: Mapped[int] = mapped_column(Id, primary_key=True)
alert_id: Mapped[int] = mapped_column(Id, ForeignKey("alerts.id", ondelete="CASCADE"))
chat_id: Mapped[int] = mapped_column(TgId)
message_id: Mapped[int] = mapped_column(BigInteger)
alert: Mapped[Alert] = relationship(back_populates="deliveries")
class LogQuery(Base):
__tablename__ = "log_queries"
id: Mapped[int] = mapped_column(Id, primary_key=True)
user_id: Mapped[int] = mapped_column(TgId, ForeignKey("users.tg_id"))
container: Mapped[str] = mapped_column(String(100))
mode: Mapped[LogMode] = mapped_column(enum(LogMode, "log_mode"))
query: Mapped[dict[str, Any]] = mapped_column(default=dict)
lines: Mapped[int] = mapped_column(Integer, default=0)
at: Mapped[datetime] = mapped_column(default=utcnow)
class AgentRequest(Base):
__tablename__ = "agent_requests"
id: Mapped[int] = mapped_column(Id, primary_key=True)
user_id: Mapped[int] = mapped_column(TgId, ForeignKey("users.tg_id"))
prompt: Mapped[str] = mapped_column(Text)
verdict: Mapped[Verdict] = mapped_column(enum(Verdict, "verdict"))
reason: Mapped[str | None] = mapped_column(Text)
log_query: Mapped[dict[str, Any] | None]
tokens: Mapped[int] = mapped_column(Integer, default=0)
at: Mapped[datetime] = mapped_column(default=utcnow)
+35
View File
@@ -0,0 +1,35 @@
"""Async engine and session factory."""
from sqlalchemy import event
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.pool import StaticPool
SessionMaker = async_sessionmaker[AsyncSession]
def make_engine(url: str) -> AsyncEngine:
if url.startswith("sqlite"):
# In-memory SQLite lives per connection: share one connection so every session sees the same DB.
kwargs = (
{"poolclass": StaticPool, "connect_args": {"check_same_thread": False}}
if ":memory:" in url or url.endswith("://")
else {}
)
engine = create_async_engine(url, **kwargs)
event.listen(engine.sync_engine, "connect", _sqlite_foreign_keys)
return engine
return create_async_engine(url, pool_pre_ping=True)
def _sqlite_foreign_keys(dbapi_conn: object, _record: object) -> None:
# SQLite ignores FOREIGN KEY / ON DELETE CASCADE unless asked per connection
cursor = dbapi_conn.cursor() # type: ignore[attr-defined]
cursor.execute("PRAGMA foreign_keys=ON")
cursor.close()
def make_sessionmaker(engine: AsyncEngine) -> SessionMaker:
return async_sessionmaker(engine, expire_on_commit=False)
__all__ = ["SessionMaker", "make_engine", "make_sessionmaker"]
+69 -18
View File
@@ -5,48 +5,99 @@ import contextlib
import logging
from aiogram import Bot, Dispatcher
from aiogram.client.session.aiohttp import AiohttpSession
from app import __version__
from app.access.service import AccessService
from app.bot.fsm import make_storage
from app.bot.middlewares.access import AccessMiddleware
from app.bot.routers.common import build_router
from app.bot.routers import commit, common, members, status
from app.config import Settings
from app.db.maintenance import cleanup_loop
from app.db.migrate import upgrade
from app.db.session import SessionMaker, make_engine, make_sessionmaker
from app.health import heartbeat
from app.logs_setup import setup_logging
from app.repo.drafts import DraftService
from app.repo.git import UploadsRepo
from app.repo.publisher import Publisher
from app.runtime.checks import Checker, load_checks
from app.runtime.docker import DockerClient
from app.runtime.snapshots import SnapshotStore, snapshot_loop
from app.texts import Texts, load_texts
log = logging.getLogger("tgit")
def build_dispatcher(settings: Settings, texts: Texts) -> Dispatcher:
admin_ids = settings.access.admin_ids
async def is_member(tg_id: int) -> bool:
return tg_id in admin_ids
dp = Dispatcher(storage=make_storage())
access = AccessMiddleware(is_member, texts)
def build_dispatcher(
texts: Texts,
sessions: SessionMaker,
access: AccessService,
repo: UploadsRepo,
commit_url_template: str = "",
docker: DockerClient | None = None,
checker: Checker | None = None,
) -> Dispatcher:
dp = Dispatcher(storage=make_storage(sessions))
gate = AccessMiddleware(access, texts)
# Outer middleware: runs before filters, so nothing — not even a stale button — reaches a handler unchecked.
dp.message.outer_middleware(access)
dp.callback_query.outer_middleware(access)
dp.include_router(build_router(texts, admin_ids))
dp.message.outer_middleware(gate)
dp.callback_query.outer_middleware(gate)
dp.include_router(members.build_router(texts, access))
dp.include_router(
commit.build_router(texts, repo, DraftService(sessions), Publisher(repo, sessions), commit_url_template)
)
dp.include_router(status.build_router(texts, docker, checker))
dp.include_router(common.build_router(texts, access))
return dp
async def run(settings: Settings) -> None:
texts = load_texts(settings.runtime.config_dir)
dp = build_dispatcher(settings, texts)
checks_cfg = load_checks(settings.runtime.config_dir)
engine = make_engine(settings.database.url)
await upgrade(engine)
sessions = make_sessionmaker(engine)
access = AccessService(sessions, settings.access)
await access.bootstrap_admins()
# No DOCKER_URL (e.g. `make run` without Docker): «Статус» answers «Docker недоступен», the rest works
docker = DockerClient(settings.docker) if settings.docker.url else None
store = SnapshotStore(sessions)
checker = Checker(docker, store, checks_cfg) if docker else None
dp = build_dispatcher(
texts, sessions, access, UploadsRepo(settings.git), settings.git.commit_url_template, docker, checker
)
rt = settings.runtime
beat = asyncio.create_task(heartbeat(rt.heartbeat_file, rt.heartbeat_interval))
bot = Bot(settings.telegram.bot_token.get_secret_value())
log.info("tgit %s starting, %d bootstrap admin(s)", __version__, len(settings.access.admin_ids))
background = [
asyncio.create_task(heartbeat(rt.heartbeat_file, rt.heartbeat_interval)),
asyncio.create_task(cleanup_loop(sessions)),
]
if docker:
background.append(asyncio.create_task(snapshot_loop(docker, store, settings.docker.snapshot_interval)))
proxy = settings.telegram.proxy_url
session = AiohttpSession(proxy=proxy.get_secret_value()) if proxy else None
bot = Bot(settings.telegram.bot_token.get_secret_value(), session=session)
log.info(
"tgit %s starting, %d bootstrap admin(s), telegram %s, docker %s",
__version__,
len(settings.access.admin_ids),
"via proxy" if proxy else "direct",
"on" if docker else "off",
)
try:
await dp.start_polling(bot, handle_signals=True)
finally:
beat.cancel()
for task in background:
task.cancel()
for task in background:
with contextlib.suppress(asyncio.CancelledError):
await beat
await task
await bot.session.close()
if docker:
await docker.aclose()
if checker:
await checker.aclose()
await engine.dispose()
log.info("tgit stopped")
View File
+107
View File
@@ -0,0 +1,107 @@
"""A member's draft: pending add/modify/delete changes until Push (SPEC C-14, C-15). One draft per member."""
from collections.abc import Callable
from dataclasses import dataclass
from datetime import UTC, datetime, timedelta
from sqlalchemy import delete, select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.db.models import ChangeOp, Draft, DraftChange, utcnow
from app.db.session import SessionMaker
MAX_CHANGES = 50
class DraftFull(Exception):
pass
@dataclass(frozen=True)
class Change:
path: str
op: ChangeOp
content: bytes | None
@dataclass(frozen=True)
class DraftView:
base_sha: str
message: str | None
changes: list[Change]
def change_for(self, path: str) -> Change | None:
return next((c for c in self.changes if c.path == path), None)
class DraftService:
def __init__(
self, sessions: SessionMaker, ttl: timedelta = timedelta(hours=24), clock: Callable[[], datetime] = utcnow
):
self._sessions = sessions
self._ttl = ttl
self._clock = clock
async def get(self, user_id: int) -> DraftView | None:
async with self._sessions() as s:
draft = await self._load(s, user_id)
if draft is None:
return None
return DraftView(
base_sha=draft.base_sha,
message=draft.message,
changes=[Change(c.path, c.op, c.content) for c in draft.changes],
)
async def put(self, user_id: int, base_sha: str, path: str, op: ChangeOp, content: bytes | None) -> None:
"""Record a change. Deleting a file that only exists in the draft just drops it from the draft."""
async with self._sessions.begin() as s:
draft = await self._load(s, user_id)
if draft is None:
draft = Draft(
user_id=user_id, base_sha=base_sha, created_at=self._clock(), expires_at=self._clock() + self._ttl
)
s.add(draft)
existing = next((c for c in draft.changes if c.path == path), None)
if existing is not None:
if op is ChangeOp.DELETE and existing.op is ChangeOp.ADD:
draft.changes.remove(existing)
return
# editing a file added in this draft keeps it an "add"
existing.op = ChangeOp.ADD if existing.op is ChangeOp.ADD and op is ChangeOp.MODIFY else op
existing.content = content
else:
if len(draft.changes) >= MAX_CHANGES:
raise DraftFull
draft.changes.append(DraftChange(path=path, op=op, content=content))
draft.expires_at = self._clock() + self._ttl
async def drop(self, user_id: int, path: str) -> None:
async with self._sessions.begin() as s:
draft = await self._load(s, user_id)
if draft is None:
return
draft.changes = [c for c in draft.changes if c.path != path]
async def set_message(self, user_id: int, message: str) -> None:
async with self._sessions.begin() as s:
draft = await self._load(s, user_id)
if draft is not None:
draft.message = message
async def clear(self, user_id: int) -> None:
async with self._sessions.begin() as s:
await s.execute(delete(Draft).where(Draft.user_id == user_id))
async def _load(self, s: AsyncSession, user_id: int) -> Draft | None:
draft = await s.scalar(select(Draft).where(Draft.user_id == user_id).options(selectinload(Draft.changes)))
if draft is not None and _aware(draft.expires_at) <= self._clock():
await s.delete(draft)
await s.flush()
return None
return draft
def _aware(value: datetime) -> datetime:
return value if value.tzinfo is not None else value.replace(tzinfo=UTC)
+168
View File
@@ -0,0 +1,168 @@
"""Local clone of the uploads repo and read-only queries against `origin/<branch>` (ARCHITECTURE §2.3).
git runs as a subprocess with an argument list — never through a shell — and with the bot's own deploy key.
"""
import asyncio
import base64
import logging
import os
import tempfile
import time
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from typing import Literal
from app.config import GitSettings
log = logging.getLogger(__name__)
EntryKind = Literal["dir", "file"]
class GitError(Exception):
pass
@dataclass(frozen=True)
class Entry:
path: str # repo-relative, "/"-separated
kind: EntryKind
size: int # bytes; 0 for directories
@property
def name(self) -> str:
return self.path.rsplit("/", 1)[-1]
@dataclass(frozen=True)
class CommitInfo:
sha: str
author: str
date: datetime
subject: str
class UploadsRepo:
def __init__(self, settings: GitSettings) -> None:
self._s = settings
# absolute: git clone runs from the parent dir, a relative path would land one level deeper
self._dir = settings.clone_dir.resolve()
self._lock = asyncio.Lock()
self._fetched_at: float | None = None
self._key_path = self._materialize_key()
@property
def branch(self) -> str:
return self._s.branch
@property
def ref(self) -> str:
return f"origin/{self._s.branch}"
@property
def path(self) -> Path:
return self._dir
def _materialize_key(self) -> Path | None:
if self._s.ssh_key_b64 is None:
return self._s.ssh_key_path
key = base64.b64decode(self._s.ssh_key_b64.get_secret_value())
fd, name = tempfile.mkstemp(prefix="tgit-ssh-") # created 0600, owned by the bot's user
with os.fdopen(fd, "wb") as f:
f.write(key if key.endswith(b"\n") else key + b"\n")
return Path(name)
def _env(self) -> dict[str, str]:
env = {k: v for k, v in os.environ.items() if k in ("PATH", "HOME", "LANG", "LC_ALL", "TMPDIR")}
env["GIT_TERMINAL_PROMPT"] = "0"
if self._key_path is not None:
known_hosts = self._dir.parent / "known_hosts"
env["GIT_SSH_COMMAND"] = (
# -F /dev/null: ignore the host's ~/.ssh/config — otherwise a personal key configured there for the
# same host gets offered too and silently masks a broken deploy key (happened in #7).
f"ssh -F /dev/null -i {self._key_path} -o IdentitiesOnly=yes -o BatchMode=yes "
f"-o StrictHostKeyChecking=accept-new -o UserKnownHostsFile={known_hosts}"
)
return env
async def git(self, *args: str, cwd: Path | None = None) -> bytes:
proc = await asyncio.create_subprocess_exec(
"git",
*args,
cwd=cwd or self._dir,
env=self._env(),
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
out, err = await asyncio.wait_for(proc.communicate(), timeout=self._s.timeout)
except TimeoutError:
proc.kill()
await proc.wait()
raise GitError(f"git {args[0]}: timeout") from None
if proc.returncode != 0:
raise GitError(f"git {args[0]}: {err.decode(errors='replace').strip()[-500:]}")
return out
async def ensure(self) -> None:
"""Clone on first use; afterwards just make sure the clone is there."""
async with self._lock:
if (self._dir / ".git").is_dir():
return
self._dir.parent.mkdir(parents=True, exist_ok=True)
log.info("cloning uploads repo into %s", self._dir)
await self.git(
"clone", "--branch", self._s.branch, self._s.uploads_url, str(self._dir), cwd=self._dir.parent
)
self._fetched_at = time.monotonic()
async def refresh(self, force: bool = False) -> None:
"""git fetch, but not more often than fetch_interval unless forced."""
await self.ensure()
async with self._lock:
now = time.monotonic()
if not force and self._fetched_at is not None and now - self._fetched_at < self._s.fetch_interval:
return
await self.git("fetch", "--prune", "origin")
self._fetched_at = now
async def head(self) -> str:
return (await self.git("rev-parse", self.ref)).decode().strip()
async def list_dir(self, path: str = "") -> list[Entry]:
"""Directories first, then files, each alphabetically."""
spec = f"{self.ref}:{path}" if path else self.ref
try:
out = await self.git("ls-tree", "-z", "-l", spec)
except GitError as e:
raise GitError(f"no such directory: /{path}") from e
entries: list[Entry] = []
for record in out.split(b"\0"):
if not record:
continue
meta, name = record.split(b"\t", 1)
_mode, kind, _obj, size = meta.split()
full = f"{path}/{name.decode()}" if path else name.decode()
if kind == b"tree":
entries.append(Entry(full, "dir", 0))
elif kind == b"blob":
entries.append(Entry(full, "file", int(size)))
return sorted(entries, key=lambda e: (e.kind != "dir", e.name.lower()))
async def read(self, path: str) -> bytes:
return await self.git("cat-file", "blob", f"{self.ref}:{path}")
async def size(self, path: str) -> int | None:
try:
return int((await self.git("cat-file", "-s", f"{self.ref}:{path}")).decode().strip())
except GitError:
return None
async def last_commit(self, path: str) -> CommitInfo | None:
out = await self.git("log", "-1", "--format=%H%x00%an%x00%aI%x00%s", self.ref, "--", path)
if not out.strip():
return None
sha, author, date, subject = out.decode().rstrip("\n").split("\0", 3)
return CommitInfo(sha=sha, author=author, date=datetime.fromisoformat(date), subject=subject)
+171
View File
@@ -0,0 +1,171 @@
"""Turns a member's draft into one commit on the uploads repo's main branch (SPEC C-20…C-25, ARCHITECTURE §2.3).
Pushes are serialized with a lock: two members pushing at once never fight over the shared working tree.
"""
import asyncio
import logging
from collections.abc import Awaitable, Callable
from dataclasses import dataclass, field
from datetime import datetime, timedelta
from enum import StrEnum
from pathlib import Path
from sqlalchemy import func, select
from app.db.models import ChangeOp, Push, User, utcnow
from app.db.session import SessionMaker
from app.repo.drafts import DraftView
from app.repo.git import GitError, UploadsRepo
log = logging.getLogger(__name__)
BOT_NAME = "tgit-bot"
BOT_EMAIL = "tgit-bot@users.noreply.tgit"
PUSHES_PER_HOUR = 20
PUSH_ATTEMPTS = 3
class PushStatus(StrEnum):
OK = "ok"
CONFLICT = "conflict" # someone changed the same files in main since the draft began
EMPTY = "empty" # the draft matches main already
LIMIT = "limit"
FAILED = "failed"
@dataclass(frozen=True)
class PushResult:
status: PushStatus
sha: str | None = None
conflicts: list[str] = field(default_factory=list)
class Publisher:
def __init__(
self,
repo: UploadsRepo,
sessions: SessionMaker,
clock: Callable[[], datetime] = utcnow,
before_push: Callable[[], Awaitable[None]] | None = None, # test hook: simulate a concurrent push
) -> None:
self._repo = repo
self._sessions = sessions
self._clock = clock
self._before_push = before_push
self._lock = asyncio.Lock()
async def push(self, member: User, draft: DraftView, message: str, *, overwrite: bool = False) -> PushResult:
if await self._pushes_last_hour(member.tg_id) >= PUSHES_PER_HOUR:
return PushResult(PushStatus.LIMIT)
async with self._lock:
try:
return await self._push(member, draft, message, overwrite)
except GitError:
log.exception("push failed")
await self._reset()
return PushResult(PushStatus.FAILED)
async def record(
self, member: User, result: PushResult, message: str, draft: DraftView, chat_id: int, message_id: int
) -> None:
assert result.sha is not None
async with self._sessions.begin() as s:
s.add(
Push(
user_id=member.tg_id,
sha=result.sha,
message=message,
paths=[c.path for c in draft.changes],
chat_id=chat_id,
message_id=message_id,
created_at=self._clock(),
)
)
async def _push(self, member: User, draft: DraftView, message: str, overwrite: bool) -> PushResult:
repo = self._repo
await repo.refresh(force=True)
if not overwrite:
conflicts = await self._conflicts(draft)
if conflicts:
return PushResult(PushStatus.CONFLICT, conflicts=conflicts)
await self._reset()
self._apply(draft)
await repo.git("add", "-A")
try:
await repo.git("diff", "--cached", "--quiet")
return PushResult(PushStatus.EMPTY) # exit 0: nothing staged
except GitError:
pass # exit 1: there are changes
handle = f"@{member.username}" if member.username else member.full_name
full_message = f"{message.strip()}\n\nTelegram-User: {handle} ({member.tg_id})"
await repo.git(
"-c", f"user.name={BOT_NAME}", "-c", f"user.email={BOT_EMAIL}", "commit", "-q", "-m", full_message
)
branch = repo.branch
for attempt in range(1, PUSH_ATTEMPTS + 1):
if self._before_push is not None:
await self._before_push()
try:
await repo.git("push", "-q", "origin", f"HEAD:{branch}")
break
except GitError:
if attempt == PUSH_ATTEMPTS:
raise
log.info("push rejected, rebasing (attempt %d)", attempt)
await repo.git("fetch", "-q", "origin")
try:
await repo.git(
"-c", f"user.name={BOT_NAME}", "-c", f"user.email={BOT_EMAIL}", "rebase", "-q", repo.ref
)
except GitError:
await repo.git("rebase", "--abort")
return PushResult(PushStatus.CONFLICT, conflicts=await self._conflicts(draft))
sha = (await repo.git("rev-parse", "HEAD")).decode().strip()
return PushResult(PushStatus.OK, sha=sha)
async def _conflicts(self, draft: DraftView) -> list[str]:
try:
out = await self._repo.git("diff", "--name-only", draft.base_sha, self._repo.ref)
except GitError:
return [c.path for c in draft.changes] # base unknown (e.g. history rewritten): be safe
changed = set(out.decode().splitlines())
return [c.path for c in draft.changes if c.path in changed]
async def _reset(self) -> None:
repo = self._repo
await repo.git("checkout", "-q", "-B", repo.branch, repo.ref)
await repo.git("reset", "-q", "--hard", repo.ref)
await repo.git("clean", "-q", "-fdx")
def _apply(self, draft: DraftView) -> None:
root = self._repo.path.resolve()
for change in draft.changes:
target = (root / change.path).resolve()
if root not in target.parents: # validated earlier; this is the last line of defence
raise GitError(f"path escapes the repo: {change.path}")
if change.op is ChangeOp.DELETE:
target.unlink(missing_ok=True)
_prune_empty_dirs(target.parent, root)
else:
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(change.content or b"")
async def _pushes_last_hour(self, user_id: int) -> int:
async with self._sessions() as s:
count = await s.scalar(
select(func.count())
.select_from(Push)
.where(Push.user_id == user_id, Push.created_at > self._clock() - timedelta(hours=1))
)
return count or 0
def _prune_empty_dirs(directory: Path, root: Path) -> None:
while directory != root and directory.is_dir() and not any(directory.iterdir()):
directory.rmdir()
directory = directory.parent
+83
View File
@@ -0,0 +1,83 @@
"""Checks for names and file contents before anything goes into a draft (SPEC C-10, C-12, C-16). No code runs here."""
import ast
import difflib
import re
from dataclasses import dataclass
MAX_FILE_BYTES = 100 * 1024
NAME = re.compile(r"^[A-Za-z0-9._-]{1,100}$")
# Bot users can't touch the CI of the uploads repo or its helper scripts (SPEC C-16, ADR-014)
PROTECTED_PREFIXES = (".sourcecraft/", ".tgit/", ".git/")
PROTECTED_FILES = {"ruff.toml"}
_FENCE = re.compile(r"^\s*```[\w+-]*[ \t]*\n(.*?)\n?```\s*$", re.DOTALL)
class ValidationError(ValueError):
pass
@dataclass(frozen=True)
class SyntaxIssue:
line: int
message: str
def check_name(name: str) -> str:
name = name.strip()
if not NAME.match(name) or name in {".", ".."}:
raise ValidationError("name")
return name
def is_protected(path: str) -> bool:
return path in PROTECTED_FILES or any(path.startswith(p) or path + "/" == p for p in PROTECTED_PREFIXES)
def join(directory: str, name: str) -> str:
return f"{directory}/{name}" if directory else name
def strip_fence(text: str) -> str:
"""Code pasted as a Telegram message often comes wrapped in ``` — keep only what's inside."""
match = _FENCE.match(text)
return match.group(1) if match else text
def check_content(data: bytes) -> str:
"""Size, encoding, NUL bytes. Returns the decoded text."""
if len(data) > MAX_FILE_BYTES:
raise ValidationError("too_big")
if b"\0" in data:
raise ValidationError("binary")
try:
return data.decode("utf-8")
except UnicodeDecodeError:
raise ValidationError("encoding") from None
def python_syntax(path: str, text: str) -> SyntaxIssue | None:
if not path.endswith(".py"):
return None
try:
ast.parse(text, filename=path)
except SyntaxError as e:
return SyntaxIssue(line=e.lineno or 0, message=e.msg)
return None
def diff_stat(old: str, new: str) -> tuple[int, int]:
added = removed = 0
for line in difflib.ndiff(old.splitlines(), new.splitlines()):
if line.startswith("+ "):
added += 1
elif line.startswith("- "):
removed += 1
return added, removed
def check_message(message: str) -> str:
message = message.strip()
if not 1 <= len(message) <= 200 or len(message.splitlines()[0]) > 72:
raise ValidationError("message")
return message
+1
View File
@@ -0,0 +1 @@
"""Containers: read-only Docker API via docker-socket-proxy, checks and snapshots (ARCHITECTURE §2.6)."""
+223
View File
@@ -0,0 +1,223 @@
"""«🩺 Запустить проверки» (SPEC S-3): independent checks per container, each with its own timeout.
A check that raises or hangs becomes ❌ with the reason; the others still run and the report is always complete.
Details are short and language-neutral (numbers, codes); the titles live in `config/texts.yaml`.
"""
import asyncio
from collections.abc import Awaitable, Callable
from dataclasses import dataclass
from datetime import datetime, timedelta
from enum import StrEnum
from pathlib import Path
import httpx
import regex
import yaml
from pydantic import BaseModel, ConfigDict, Field, field_validator
from app.db.models import utcnow
from app.runtime.docker import Details, DockerClient, Stats
from app.runtime.snapshots import SnapshotStore
REGEX_TIMEOUT = 1.0 # seconds per log line: a pathological pattern can't stall the bot
class ChecksConfig(BaseModel):
model_config = ConfigDict(extra="forbid", frozen=True)
window: int = Field(default=300, gt=0)
cpu_pct: float = Field(default=90, gt=0)
mem_pct: float = Field(default=90, gt=0, le=100)
error_pattern: str
http_timeout: float = Field(default=5, gt=0)
check_timeout: float = Field(default=15, gt=0)
@field_validator("error_pattern")
@classmethod
def _compiles(cls, value: str) -> str:
try:
regex.compile(value)
except regex.error as e: # not a ValueError: pydantic wouldn't turn it into a validation error
raise ValueError(f"error_pattern: {e}") from e
return value
def load_checks(config_dir: Path) -> ChecksConfig:
with (config_dir / "checks.yaml").open(encoding="utf-8") as f:
return ChecksConfig.model_validate(yaml.safe_load(f))
class CheckName(StrEnum):
RUNNING = "running"
HEALTH = "health"
RESTARTS = "restarts"
CPU = "cpu"
MEM = "mem"
LOG_ERRORS = "log_errors"
HTTP = "http"
class Verdict(StrEnum):
OK = "✅"
FAIL = "❌"
SKIP = "➖" # not applicable: no healthcheck, not running, no health_url
@dataclass(frozen=True)
class CheckResult:
name: CheckName
verdict: Verdict
detail: str = ""
@dataclass(frozen=True)
class ContainerReport:
details: Details
results: list[CheckResult]
@property
def ok(self) -> bool:
return all(r.verdict is not Verdict.FAIL for r in self.results)
@dataclass(frozen=True)
class Context:
details: Details
docker: DockerClient
store: SnapshotStore
cfg: ChecksConfig
http: httpx.AsyncClient
now: datetime
stats: Stats | None # None when not running
@property
def since(self) -> datetime:
return self.now - timedelta(seconds=self.cfg.window)
async def check_running(ctx: Context) -> CheckResult:
d = ctx.details
if d.state == "running":
return CheckResult(CheckName.RUNNING, Verdict.OK, d.state)
detail = f"{d.state}, exit {d.exit_code}" + (", OOM" if d.oom_killed else "")
return CheckResult(CheckName.RUNNING, Verdict.FAIL, detail)
async def check_health(ctx: Context) -> CheckResult:
health = ctx.details.health
if health is None:
return CheckResult(CheckName.HEALTH, Verdict.SKIP, "no healthcheck")
return CheckResult(CheckName.HEALTH, Verdict.OK if health == "healthy" else Verdict.FAIL, health)
async def check_restarts(ctx: Context) -> CheckResult:
now_count = ctx.details.restart_count
before = await ctx.store.oldest_since(ctx.details.name, ctx.since)
if before is None:
return CheckResult(CheckName.RESTARTS, Verdict.SKIP, f"{now_count}, no history")
grown = now_count - before.restart_count
if grown > 0:
return CheckResult(CheckName.RESTARTS, Verdict.FAIL, f"{before.restart_count} → {now_count}")
return CheckResult(CheckName.RESTARTS, Verdict.OK, str(now_count))
async def check_cpu(ctx: Context) -> CheckResult:
if ctx.stats is None:
return CheckResult(CheckName.CPU, Verdict.SKIP)
pct = ctx.stats.cpu_pct
verdict = Verdict.OK if pct < ctx.cfg.cpu_pct else Verdict.FAIL
return CheckResult(CheckName.CPU, verdict, f"{pct:.1f}% / {ctx.cfg.cpu_pct:g}%")
async def check_mem(ctx: Context) -> CheckResult:
if ctx.stats is None:
return CheckResult(CheckName.MEM, Verdict.SKIP)
pct = ctx.stats.mem_pct
verdict = Verdict.OK if pct < ctx.cfg.mem_pct else Verdict.FAIL
return CheckResult(CheckName.MEM, verdict, f"{pct:.1f}% / {ctx.cfg.mem_pct:g}%")
def count_errors(text: str, pattern: str) -> tuple[int, str]:
"""Matching lines and the first of them (trimmed)."""
compiled = regex.compile(pattern)
hits, first = 0, ""
for line in text.splitlines():
try:
matched = compiled.search(line, timeout=REGEX_TIMEOUT) is not None
except TimeoutError:
matched = False
if matched:
hits += 1
first = first or line.strip()[:120]
return hits, first
async def check_log_errors(ctx: Context) -> CheckResult:
text = await ctx.docker.logs(ctx.details.id, since=ctx.since)
hits, first = await asyncio.to_thread(count_errors, text, ctx.cfg.error_pattern)
if hits:
return CheckResult(CheckName.LOG_ERRORS, Verdict.FAIL, f"{hits}: {first}")
return CheckResult(CheckName.LOG_ERRORS, Verdict.OK, "0")
async def check_http(ctx: Context) -> CheckResult:
url = ctx.details.health_url
if url is None:
return CheckResult(CheckName.HTTP, Verdict.SKIP, "no health_url")
try:
response = await ctx.http.get(url, timeout=ctx.cfg.http_timeout)
except httpx.HTTPError as e:
return CheckResult(CheckName.HTTP, Verdict.FAIL, type(e).__name__)
ok = response.status_code < 400
return CheckResult(CheckName.HTTP, Verdict.OK if ok else Verdict.FAIL, f"HTTP {response.status_code}")
Check = Callable[[Context], Awaitable[CheckResult]]
CHECKS: list[tuple[CheckName, Check]] = [
(CheckName.RUNNING, check_running),
(CheckName.HEALTH, check_health),
(CheckName.RESTARTS, check_restarts),
(CheckName.CPU, check_cpu),
(CheckName.MEM, check_mem),
(CheckName.LOG_ERRORS, check_log_errors),
(CheckName.HTTP, check_http),
]
async def _guarded(name: CheckName, check: Check, ctx: Context) -> CheckResult:
try:
return await asyncio.wait_for(check(ctx), timeout=ctx.cfg.check_timeout)
except TimeoutError:
return CheckResult(name, Verdict.FAIL, "timeout")
except Exception as e: # one broken check must not spoil the report
return CheckResult(name, Verdict.FAIL, f"{type(e).__name__}: {str(e)[:80]}")
class Checker:
def __init__(
self, docker: DockerClient, store: SnapshotStore, cfg: ChecksConfig, http: httpx.AsyncClient | None = None
) -> None:
self._docker = docker
self._store = store
self._cfg = cfg
self._http = http or httpx.AsyncClient(follow_redirects=False)
async def check(self, container_id: str, now: datetime | None = None) -> ContainerReport:
details = await self._docker.inspect(container_id)
stats = None
if details.state == "running":
try:
stats = await asyncio.wait_for(self._docker.stats(container_id), timeout=self._cfg.check_timeout)
except Exception: # cpu/mem become ➖, the rest still runs
stats = None
ctx = Context(details, self._docker, self._store, self._cfg, self._http, now or utcnow(), stats)
results = await asyncio.gather(*(_guarded(name, fn, ctx) for name, fn in CHECKS))
return ContainerReport(details, list(results))
async def check_all(self, now: datetime | None = None) -> list[ContainerReport]:
containers = await self._docker.containers()
return list(await asyncio.gather(*(self.check(c.id, now) for c in containers)))
async def aclose(self) -> None:
await self._http.aclose()
+283
View File
@@ -0,0 +1,283 @@
"""Read-only Docker Engine API client. The bot talks to docker-socket-proxy (GET only), never to the socket itself.
Only containers carrying the managed label are visible: the host also runs other projects (Vobot, meditron),
and an id outside the list is refused before any request goes out.
"""
import json
import re
import struct
from dataclasses import dataclass
from datetime import UTC, datetime
from enum import StrEnum
from types import TracebackType
from typing import Any, Self
import httpx
from app.config import DockerSettings
SHA_LABEL = "tgit.sha"
DEPLOYED_LABEL = "tgit.deployed_at"
HEALTH_URL_LABEL = "tgit.health_url"
SERVICE_LABEL = "com.docker.compose.service"
_FRACTION = re.compile(r"(\.\d{6})\d+") # Docker gives nanoseconds, datetime takes microseconds
class DockerError(Exception):
pass
class Light(StrEnum):
"""S-1: 🟢 healthy / 🟡 starting / 🔴 unhealthy or crashed / ⚪ stopped."""
GREEN = "🟢"
YELLOW = "🟡"
RED = "🔴"
WHITE = "⚪"
def parse_time(value: str | None) -> datetime | None:
if not value or value.startswith("0001-"): # Docker's zero time: never started / no deploy label
return None
try:
return datetime.fromisoformat(_FRACTION.sub(r"\1", value).replace("Z", "+00:00")).astimezone(UTC)
except ValueError:
return None
def light(state: str, health: str | None, exit_code: int = 0) -> Light:
if state == "running":
if health == "unhealthy":
return Light.RED
if health == "starting":
return Light.YELLOW
return Light.GREEN
if state in ("restarting", "dead") or (state == "exited" and exit_code != 0):
return Light.RED
if state == "created":
return Light.YELLOW
return Light.WHITE
@dataclass(frozen=True)
class Container:
"""A list entry (GET /containers/json): enough for the overview, no extra request per container."""
id: str
name: str
service: str
image: str
state: str # created | running | paused | restarting | removing | exited | dead
health: str | None # starting | healthy | unhealthy | None — no healthcheck
exit_code: int
labels: dict[str, str]
@property
def short_id(self) -> str:
return self.id[:12]
@property
def light(self) -> Light:
return light(self.state, self.health, self.exit_code)
_HEALTH_IN_STATUS = re.compile(r"\((healthy|unhealthy|health: starting)\)")
_EXIT_IN_STATUS = re.compile(r"^Exited \((-?\d+)\)")
def _health_from_status(status: str) -> str | None:
"""`Up 2 hours (healthy)` → healthy. The list endpoint has no structured health field."""
m = _HEALTH_IN_STATUS.search(status)
if m is None:
return None
return "starting" if m.group(1) == "health: starting" else m.group(1)
@dataclass(frozen=True)
class Details:
"""GET /containers/{id}/json — the card (S-2) and most checks."""
id: str
name: str
service: str
image: str
state: str
health: str | None
exit_code: int
restart_count: int
started_at: datetime | None
finished_at: datetime | None
oom_killed: bool
labels: dict[str, str]
@property
def light(self) -> Light:
return light(self.state, self.health, self.exit_code)
@property
def sha(self) -> str | None:
return self.labels.get(SHA_LABEL) or None
@property
def deployed_at(self) -> datetime | None:
return parse_time(self.labels.get(DEPLOYED_LABEL))
@property
def health_url(self) -> str | None:
return self.labels.get(HEALTH_URL_LABEL) or None
@dataclass(frozen=True)
class Stats:
cpu_pct: float # of one CPU × online CPUs, as `docker stats` shows it
mem_bytes: int
mem_limit: int # the cgroup limit; the host's RAM when the container has none
@property
def mem_pct(self) -> float:
return 100.0 * self.mem_bytes / self.mem_limit if self.mem_limit else 0.0
def parse_stats(data: dict[str, Any]) -> Stats:
cpu, pre = data.get("cpu_stats") or {}, data.get("precpu_stats") or {}
cpu_delta = (cpu.get("cpu_usage") or {}).get("total_usage", 0) - (pre.get("cpu_usage") or {}).get("total_usage", 0)
sys_delta = cpu.get("system_cpu_usage", 0) - pre.get("system_cpu_usage", 0)
cpus = cpu.get("online_cpus") or len((cpu.get("cpu_usage") or {}).get("percpu_usage") or []) or 1
cpu_pct = 100.0 * cpu_delta / sys_delta * cpus if cpu_delta > 0 and sys_delta > 0 else 0.0
mem = data.get("memory_stats") or {}
extra = mem.get("stats") or {}
# Page cache isn't pressure: cgroup v2 reports inactive_file, v1 total_inactive_file (docker CLI does the same)
cache = extra.get("inactive_file", extra.get("total_inactive_file", 0))
usage = max(int(mem.get("usage", 0)) - int(cache), 0)
return Stats(cpu_pct=round(cpu_pct, 1), mem_bytes=usage, mem_limit=int(mem.get("limit", 0)))
def demux_logs(raw: bytes) -> str:
"""Split Docker's multiplexed stream (8-byte header per frame) — containers without a TTY send it this way."""
if raw[:1] not in (b"\x00", b"\x01", b"\x02") or len(raw) < 8 or raw[1:4] != b"\x00\x00\x00":
return raw.decode(errors="replace") # TTY container: plain bytes
out: list[bytes] = []
pos = 0
while pos + 8 <= len(raw):
(size,) = struct.unpack(">I", raw[pos + 4 : pos + 8])
out.append(raw[pos + 8 : pos + 8 + size])
pos += 8 + size
return b"".join(out).decode(errors="replace")
class DockerClient:
def __init__(self, settings: DockerSettings, transport: httpx.AsyncBaseTransport | None = None) -> None:
self._s = settings
self._http = httpx.AsyncClient(base_url=settings.url, timeout=settings.timeout, transport=transport)
self._known: set[str] = set() # ids from the last listing: the only ones card/logs/stats may touch
async def __aenter__(self) -> Self:
return self
async def __aexit__(
self, exc_type: type[BaseException] | None, exc: BaseException | None, tb: TracebackType | None
) -> None:
await self.aclose()
async def aclose(self) -> None:
await self._http.aclose()
async def containers(self) -> list[Container]:
"""All managed containers, stopped ones too, sorted by name."""
filters = json.dumps({"label": [self._s.label]})
data = await self._get_json("/containers/json", params={"all": "1", "filters": filters})
result = []
for c in data:
labels = c.get("Labels") or {}
names = c.get("Names") or [c["Id"][:12]]
result.append(
Container(
id=c["Id"],
name=names[0].lstrip("/"),
service=labels.get(SERVICE_LABEL, ""),
image=c.get("Image", ""),
state=c.get("State", ""),
health=_health_from_status(c.get("Status", "")),
exit_code=int(m.group(1)) if (m := _EXIT_IN_STATUS.match(c.get("Status", ""))) else 0,
labels=labels,
)
)
self._known = {c.id for c in result}
return sorted(result, key=lambda c: c.name)
async def resolve(self, short_id: str) -> str:
"""Full id of a managed container by its 12-char prefix (callback data carries only that)."""
if not self._known:
await self.containers()
for full in self._known:
if full.startswith(short_id):
return full
await self.containers() # maybe recreated since the last listing
for full in self._known:
if full.startswith(short_id):
return full
raise DockerError(f"container {short_id} is not managed by tgit")
async def inspect(self, container_id: str) -> Details:
self._guard(container_id)
d = await self._get_json(f"/containers/{container_id}/json")
state = d.get("State") or {}
labels = (d.get("Config") or {}).get("Labels") or {}
return Details(
id=d["Id"],
name=d.get("Name", "").lstrip("/"),
service=labels.get(SERVICE_LABEL, ""),
image=(d.get("Config") or {}).get("Image", ""),
state=state.get("Status", ""),
health=(state.get("Health") or {}).get("Status"),
exit_code=int(state.get("ExitCode", 0)),
restart_count=int(d.get("RestartCount", 0)),
started_at=parse_time(state.get("StartedAt")),
finished_at=parse_time(state.get("FinishedAt")),
oom_killed=bool(state.get("OOMKilled", False)),
labels=labels,
)
async def stats(self, container_id: str) -> Stats:
"""One sample with a previous one to diff against (stream=false waits ~1 s for it)."""
self._guard(container_id)
return parse_stats(await self._get_json(f"/containers/{container_id}/stats", params={"stream": "false"}))
async def logs(self, container_id: str, since: datetime | None = None, tail: int | None = None) -> str:
self._guard(container_id)
params = {"stdout": "1", "stderr": "1"}
if since is not None:
params["since"] = str(int(since.timestamp()))
if tail is not None:
params["tail"] = str(tail)
response = await self._request(f"/containers/{container_id}/logs", params)
raw = response.content[: self._s.max_log_bytes]
return demux_logs(raw)
async def ping(self) -> bool:
try:
return (await self._request("/_ping", {})).text == "OK"
except DockerError:
return False
def _guard(self, container_id: str) -> None:
if container_id not in self._known:
raise DockerError(f"container {container_id[:12]} is not managed by tgit")
async def _get_json(self, path: str, params: dict[str, str] | None = None) -> Any:
response = await self._request(path, params or {})
try:
return response.json()
except ValueError as e:
raise DockerError(f"{path}: not JSON") from e
async def _request(self, path: str, params: dict[str, str]) -> httpx.Response:
try:
response = await self._http.get(path, params=params)
except httpx.TransportError as e: # timeouts included
raise DockerError(f"docker unreachable: {type(e).__name__}") from e
if response.status_code >= 400:
raise DockerError(f"{path}: HTTP {response.status_code} {response.text[:200]}")
return response
+87
View File
@@ -0,0 +1,87 @@
"""Every 15 s: restart count, CPU and RAM of each running managed container → `container_snapshots`.
The history answers «are restarts growing over the last 5 minutes» (S-3) and feeds alerts (#15).
Old rows are removed by `app.db.maintenance`.
"""
import asyncio
import logging
from datetime import datetime
from sqlalchemy import select
from app.db.models import ContainerSnapshot, utcnow
from app.db.session import SessionMaker
from app.runtime.docker import DockerClient, DockerError
log = logging.getLogger(__name__)
class SnapshotStore:
def __init__(self, sessions: SessionMaker) -> None:
self._sessions = sessions
async def add(self, rows: list[ContainerSnapshot]) -> None:
if rows:
async with self._sessions.begin() as session:
session.add_all(rows)
async def oldest_since(self, container: str, since: datetime) -> ContainerSnapshot | None:
async with self._sessions() as session:
return await session.scalar(
select(ContainerSnapshot)
.where(ContainerSnapshot.container == container, ContainerSnapshot.at >= since)
.order_by(ContainerSnapshot.at)
.limit(1)
)
async def latest(self, container: str) -> ContainerSnapshot | None:
async with self._sessions() as session:
return await session.scalar(
select(ContainerSnapshot)
.where(ContainerSnapshot.container == container)
.order_by(ContainerSnapshot.at.desc())
.limit(1)
)
async def _snapshot(docker: DockerClient, container_id: str, at: datetime) -> ContainerSnapshot | None:
try:
details, stats = await asyncio.gather(docker.inspect(container_id), docker.stats(container_id))
except DockerError as e:
log.warning("snapshot %s: %s", container_id[:12], e)
return None
return ContainerSnapshot(
container=details.name,
at=at,
restart_count=details.restart_count,
cpu_pct=stats.cpu_pct,
mem_pct=round(stats.mem_pct, 1),
health=details.health,
)
async def collect_once(docker: DockerClient, store: SnapshotStore, now: datetime | None = None) -> int:
at = now or utcnow()
running = [c for c in await docker.containers() if c.state == "running"]
rows = await asyncio.gather(*(_snapshot(docker, c.id, at) for c in running))
snapshots = [r for r in rows if r is not None]
await store.add(snapshots)
return len(snapshots)
async def snapshot_loop(docker: DockerClient, store: SnapshotStore, interval: float) -> None:
failing = False
while True:
try:
await collect_once(docker, store)
if failing:
log.info("snapshots: docker reachable again")
failing = False
except DockerError as e:
if not failing: # one line per outage, not one every 15 s
log.warning("snapshots: %s", e)
failing = True
except Exception: # a broken round must not kill the bot
log.exception("snapshots failed")
await asyncio.sleep(interval)
View File
+28
View File
@@ -0,0 +1,28 @@
"""Debug CLI: `python -m app.sourcecraft.cli runs [repo]` — recent CI runs with their commit and failed cubes."""
import asyncio
import sys
from app.config import SourceCraftSettings
from app.sourcecraft.client import SourceCraftClient
async def runs(repo: str | None) -> None:
settings = SourceCraftSettings()
repo = repo or settings.uploads_repo
async with SourceCraftClient(settings) as client:
for run in await client.list_runs(repo, page_size=10):
full = await client.get_run(repo, run.slug)
sha = await client.run_commit_sha(repo, full)
failed = ", ".join(f"{t}/{c.slug}" for _, t, c in full.failed_cubes()) or "-"
print(f"{run.slug} {run.event_type:<8} {run.status:<10} sha={sha or '?':<40.12} failed={failed}")
def main() -> None:
if len(sys.argv) < 2 or sys.argv[1] != "runs":
sys.exit("usage: python -m app.sourcecraft.cli runs [org/repo]")
asyncio.run(runs(sys.argv[2] if len(sys.argv) > 2 else None))
if __name__ == "__main__":
main()
+140
View File
@@ -0,0 +1,140 @@
"""SourceCraft REST API client (ARCHITECTURE §2.4). The API drops TLS connections now and then: every call retries.
Runs don't carry a commit sha, so the uploads CI prints it from its first cube (`check/meta`, ADR-014);
`run_commit_sha` reads it back.
"""
import asyncio
import logging
import re
from collections.abc import Awaitable, Callable
from types import TracebackType
from typing import Any, Self
import httpx
from app.config import SourceCraftSettings
from app.sourcecraft.models import CubeLog, Run
log = logging.getLogger(__name__)
META_TASK = "check"
META_CUBE = "meta"
_SHA_LINE = re.compile(r"^TGIT_COMMIT_SHA=([0-9a-f]{40})\s*$", re.MULTILINE)
Sleep = Callable[[float], Awaitable[None]]
class SourceCraftError(Exception):
def __init__(self, status: int, code: str, message: str) -> None:
super().__init__(f"{status} {code}: {message}")
self.status = status
self.code = code
self.message = message
class SourceCraftClient:
def __init__(
self,
settings: SourceCraftSettings,
transport: httpx.AsyncBaseTransport | None = None,
sleep: Sleep = asyncio.sleep,
) -> None:
self._settings = settings
self._sleep = sleep
self._http = httpx.AsyncClient(
base_url=settings.api_url,
headers={"Authorization": f"Bearer {settings.token.get_secret_value()}"},
timeout=settings.timeout,
transport=transport,
)
async def __aenter__(self) -> Self:
return self
async def __aexit__(
self, exc_type: type[BaseException] | None, exc: BaseException | None, tb: TracebackType | None
) -> None:
await self.aclose()
async def aclose(self) -> None:
await self._http.aclose()
def run_url(self, repo: str, run: str) -> str:
return self._settings.run_url_template.format(repo=repo, run=run)
async def list_runs(self, repo: str, page_size: int = 20) -> list[Run]:
"""Newest first. One page is enough for the watcher: it only looks for runs of recent pushes."""
data = await self._get(f"/repos/{repo}/cicd/runs", params={"page_size": page_size})
return [Run.model_validate(r) for r in data.get("runs", [])]
async def get_run(self, repo: str, run: str) -> Run:
return Run.model_validate(await self._get(f"/repos/{repo}/cicd/runs/{run}"))
async def get_cube_logs(self, repo: str, run: str, workflow: str, task: str, cube: str) -> CubeLog:
"""Concatenate log pages in order; a page that isn't complete yet is the last readable one."""
parts: list[str] = []
done = False
page = 1
while True:
try:
data = await self._get(f"/repos/{repo}/cicd/logs/{run}/{workflow}/{task}/{cube}", params={"page": page})
except SourceCraftError as e:
if e.status == 404 and page > 1:
break
raise
parts.append(data.get("logs", ""))
done = bool(data.get("done", False))
if not data.get("page_complete", False):
break
page += 1
return CubeLog(text="".join(parts), done=done)
async def run_commit_sha(self, repo: str, run: Run) -> str | None:
"""The commit a run was started for, read from the meta cube's log (None if not printed yet)."""
for workflow in run.workflows:
if run.task(workflow.slug, META_TASK) is None:
continue
try:
logs = await self.get_cube_logs(repo, run.slug, workflow.slug, META_TASK, META_CUBE)
except SourceCraftError as e:
if e.status == 404:
return None
raise
match = _SHA_LINE.search(logs.text)
return match.group(1) if match else None
return None
async def _get(self, path: str, params: dict[str, Any] | None = None) -> dict[str, Any]:
attempts = self._settings.max_attempts
for attempt in range(1, attempts + 1):
try:
response = await self._http.get(path, params=params)
except httpx.TransportError as e:
if attempt == attempts:
raise
log.warning("sourcecraft %s: %s, retry %d/%d", path, type(e).__name__, attempt, attempts - 1)
else:
if response.status_code < 500:
return self._parse(response)
if attempt == attempts:
return self._parse(response)
log.warning("sourcecraft %s: HTTP %d, retry %d/%d", path, response.status_code, attempt, attempts - 1)
await self._sleep(min(0.5 * 2 ** (attempt - 1), 8.0))
raise AssertionError("unreachable")
@staticmethod
def _parse(response: httpx.Response) -> dict[str, Any]:
try:
body = response.json()
except ValueError:
body = {}
if response.status_code >= 400:
raise SourceCraftError(
response.status_code,
str(body.get("error_code", "")),
str(body.get("message", response.text[:200])),
)
if not isinstance(body, dict):
raise SourceCraftError(response.status_code, "BadResponse", "expected a JSON object")
return body
+113
View File
@@ -0,0 +1,113 @@
"""Typed SourceCraft CI/CD API responses (only the fields the bot uses)."""
from collections.abc import Iterator
from datetime import datetime
from enum import StrEnum
from pydantic import BaseModel, ConfigDict, Field, field_validator
class RunStatus(StrEnum):
CREATED = "created"
PREPARED = "prepared"
PROCESSING = "processing"
SUCCESS = "success"
FAILED = "failed"
CANCELED = "canceled"
TIMEOUT = "timeout"
SKIPPED = "skipped"
AWAITING_APPROVAL = "awaiting_approval"
REJECTED = "rejected"
@property
def finished(self) -> bool:
return self in _FINISHED
_FINISHED = {
RunStatus.SUCCESS,
RunStatus.FAILED,
RunStatus.CANCELED,
RunStatus.TIMEOUT,
RunStatus.SKIPPED,
RunStatus.REJECTED,
}
class EventType(StrEnum):
PUSH = "push"
PR_UPDATE = "pr_update"
MANUAL = "manual"
RESTART = "restart"
SCHEDULE = "schedule"
REPOSITORY_EVENT = "repository_event"
class _Model(BaseModel):
model_config = ConfigDict(extra="ignore", frozen=True)
class Dates(_Model):
created_at: datetime | None = None
started_at: datetime | None = None
finished_at: datetime | None = None
@field_validator("created_at", "started_at", "finished_at", mode="after")
@classmethod
def _epoch_is_unset(cls, value: datetime | None) -> datetime | None:
# The API reports "not yet" as 1970-01-01T00:00:00Z
return None if value is not None and value.year == 1970 else value
class Cube(_Model):
slug: str
status: RunStatus
dates: Dates = Field(default_factory=Dates)
@property
def is_service(self) -> bool:
"""Platform steps (#git-clone, #prepare-image-for-…): not in our config, logs not readable."""
return self.slug.startswith("#")
class Task(_Model):
slug: str
status: RunStatus
cubes: list[Cube] = []
class Workflow(_Model):
slug: str
status: RunStatus
tasks: list[Task] = []
class Run(_Model):
slug: str
status: RunStatus
event_type: EventType
dates: Dates = Field(default_factory=Dates)
workflows: list[Workflow] = []
error_messages: list[str] = []
def task(self, workflow: str, task: str) -> Task | None:
for wf in self.workflows:
if wf.slug == workflow:
return next((t for t in wf.tasks if t.slug == task), None)
return None
def cubes(self) -> Iterator[tuple[str, str, Cube]]:
"""(workflow, task, cube) for every user-defined cube."""
for wf in self.workflows:
for t in wf.tasks:
for c in t.cubes:
if not c.is_service:
yield wf.slug, t.slug, c
def failed_cubes(self) -> list[tuple[str, str, Cube]]:
return [entry for entry in self.cubes() if entry[2].status is RunStatus.FAILED]
class CubeLog(_Model):
text: str
done: bool
+117 -1
View File
@@ -24,6 +24,7 @@ class MenuTexts(_Strict):
class StartTexts(_Strict):
welcome: str
no_access: str
invite_rejected: str
class HelpTexts(_Strict):
@@ -32,18 +33,133 @@ class HelpTexts(_Strict):
class SectionTexts(_Strict):
commit: str
status: str
logs: str
alerts: str
members: str
cancelled: str
class MembersTexts(_Strict):
title: str
line: str
never: str
invite: str
revoke: str
confirm: str
confirm_yes: str
confirm_no: str
revoked: str
not_revoked: str
invite_caption: str
class CommitTexts(_Strict):
title: str
empty: str
page: str
up: str
prev: str
next: str
dir_button: str
file_button: str
new_file: str
new_dir: str
file_card: str
last_commit: str
no_commits: str
view: str
replace: str
delete: str
to_dir: str
soon: str
unavailable: str
sent_as_file: str
ask_file_name: str
ask_dir_name: str
bad_name: str
exists: str
protected: str
ask_content: str
too_big: str
binary: str
encoding: str
syntax: str
fix: str
force: str
added: str
modified: str
deleted: str
draft_button: str
draft_title: str
draft_empty: str
unstage: str
draft_full: str
in_draft_new: str
in_draft_modified: str
in_draft_deleted: str
mark_new: str
mark_modified: str
mark_deleted: str
op_add: str
op_modify: str
op_delete: str
message_button: str
message_current: str
message_missing: str
ask_message: str
bad_message: str
push_button: str
commit_button: str
edit_message: str
push_confirm: str
push_go: str
pushing: str
pushed: str
ci_pending: str
conflict: str
overwrite: str
push_empty: str
push_limit: str
push_failed: str
class StatusTexts(_Strict):
title: str
line: str
empty: str
unavailable: str
card: str
none: str
no_limit: str
check_button: str
refresh: str
to_list: str
to_card: str
checking: str
report_title: str
report_container: str
report_line: str
check_running: str
check_health: str
check_restarts: str
check_cpu: str
check_mem: str
check_log_errors: str
check_http: str
days: str
hours: str
minutes: str
seconds: str
class Texts(_Strict):
menu: MenuTexts
start: StartTexts
help: HelpTexts
section: SectionTexts
members: MembersTexts
commit: CommitTexts
status: StatusTexts
def load_texts(config_dir: Path) -> Texts:
+13
View File
@@ -0,0 +1,13 @@
# Проверки раздела «🩺 Статус» (SPEC S-3). Читает app/runtime/checks.py; неизвестный ключ — ошибка при старте.
# Окно для «рестарты не растут» и «нет ошибок в логе», секунды
window: 300
# Пороги, % от лимита контейнера (CPU — от числа доступных ядер × 100)
cpu_pct: 90
mem_pct: 90
# Строка лога считается ошибкой, если совпала с шаблоном (синтаксис модуля regex)
error_pattern: '(?i)\b(error|exception|traceback|critical|fatal|panic)\b|"level":\s*"(ERROR|CRITICAL)"'
# HTTP-проба health_url (метка tgit.health_url): таймаут, секунды
http_timeout: 5
# Таймаут одной проверки — упавшая или зависшая проверка не роняет отчёт
check_timeout: 15
+134 -1
View File
@@ -19,6 +19,7 @@ start:
📜 Логи — хвост, свой grep или вопрос агенту
🚨 Алерты — падения, перегрузка, список
no_access: "Доступ к боту — только по приглашению. Попросите QR-код у администратора."
invite_rejected: "Приглашение недействительно: истекло, уже использовано или ссылка неверная. Попросите новое у администратора."
help:
text: |-
@@ -29,8 +30,140 @@ help:
section:
commit: "📝 Коммит\n\nРаздел в разработке."
status: "🩺 Статус\n\nРаздел в разработке."
logs: "📜 Логи\n\nРаздел в разработке."
alerts: "🚨 Алерты\n\nРаздел в разработке."
members: "👥 Участники\n\nРаздел в разработке."
cancelled: "Отменено. Главное меню — кнопками внизу."
members:
title: "👥 Участники: {count}"
line: "{icon} {name}{username} · с {joined} · был(а) {seen}"
never: "—"
invite: "➕ Пригласить"
revoke: "🚫 {name}"
confirm: "Отозвать доступ у {name}? Его черновик удалится, кнопки перестанут работать."
confirm_yes: "Да, отозвать"
confirm_no: "Нет"
revoked: "Доступ отозван: {name}."
not_revoked: "Нельзя отозвать: администратор или уже без доступа."
invite_caption: |-
Приглашение в tgit — отсканируйте QR или откройте ссылку:
{link}
Одноразовое, действует до {expires} UTC.
commit:
title: "📝 Коммит · /{path}"
empty: "Папка пуста."
page: "Стр. {page} из {pages}"
up: "⬆️ Вверх"
prev: "◀️"
next: "▶️"
dir_button: "📁 {name}"
file_button: "📄 {name} · {size}"
new_file: "➕ Новый файл"
new_dir: "📁 Новая папка"
file_card: |-
📄 /{path}
Размер: {size}
last_commit: |-
Последний коммит: {sha} · {author} · {date}
{subject}
no_commits: "Коммитов по файлу нет."
view: "👁 Показать"
replace: "✏️ Заменить"
delete: "🗑 Удалить"
to_dir: "⬅️ К папке"
soon: "Будет в следующей версии."
unavailable: "Репозиторий загрузок недоступен, попробуйте через минуту."
sent_as_file: "Файл большой или бинарный — отправил документом."
ask_file_name: "Имя нового файла в /{dir} (латиница, цифры, «.», «_», «-»):"
ask_dir_name: "Имя новой папки в /{dir}:"
bad_name: "Недопустимое имя: латиница, цифры, «.», «_», «-», до 100 символов. Пришлите другое."
exists: "Такое имя уже есть в этой папке. Пришлите другое."
protected: "Это служебный путь репозитория — из бота его менять нельзя."
ask_content: "Пришлите содержимое /{path} — текстом сообщения или файлом (до 100 КБ)."
too_big: "Больше 100 КБ — не принимаю."
binary: "Похоже на бинарный файл — принимаю только текст."
encoding: "Файл не в кодировке UTF-8."
syntax: "⚠️ Синтаксическая ошибка, строка {line}: {message}"
fix: "✏️ Исправить"
force: "Всё равно добавить"
added: "🆕 /{path} — добавлен в черновик."
modified: "✏️ /{path} — в черновике: +{added} −{removed}."
deleted: "🗑 /{path} — в черновике на удаление."
draft_button: "🧾 Черновик · {count}"
draft_title: "🧾 Черновик · изменений: {count}"
draft_empty: "Черновик пуст."
unstage: "↩️ убрать {name}"
draft_full: "В черновике уже 50 изменений — запушьте или уберите лишнее."
in_draft_new: "В черновике: новый файл."
in_draft_modified: "В черновике: изменён."
in_draft_deleted: "В черновике: удалён."
mark_new: "🆕 "
mark_modified: "✏️ "
mark_deleted: "🗑 "
op_add: "A"
op_modify: "M"
op_delete: "D"
message_button: "💬 Сообщение коммита"
message_current: "💬 {message}"
message_missing: "💬 Сообщение коммита не задано."
ask_message: "Сообщение коммита: до 200 символов, первая строка — до 72."
bad_message: "Сообщение: 1–200 символов, первая строка — до 72. Пришлите другое."
push_button: "🚀 Push"
commit_button: "🚀 Закоммитить · {count}"
edit_message: "✏️ Изменить сообщение"
push_confirm: |-
Запушить в main?
{summary}
💬 {message}
push_go: "✅ Запушить"
pushing: "⏳ Пушу…"
pushed: "✅ Запушено: {sha} — {message}"
ci_pending: "⏳ CI…"
conflict: |-
⚠️ Пока вы готовили черновик, в main изменились те же файлы:
{paths}
Перезаписать их вашей версией?
overwrite: "Перезаписать"
push_empty: "В main уже всё так же — коммитить нечего. Черновик очищен."
push_limit: "Лимит — 20 пушей в час. Попробуйте позже."
push_failed: "Не удалось запушить: репозиторий недоступен. Черновик сохранён — попробуйте позже."
status:
title: "🩺 Статус · {count} конт. · {time} UTC"
line: "{light} {name} · {state}"
empty: "Контейнеров tgit не найдено."
unavailable: "Docker недоступен, попробуйте через минуту."
card: |-
{light} {name}
Сервис: {service} · образ: {image}
Состояние: {state} · health: {health}
Аптайм: {uptime} · рестартов: {restarts}
CPU: {cpu} · RAM: {mem} из {limit} ({mem_pct})
Ревизия: {sha} · деплой: {deployed}
none: "—"
no_limit: "без лимита"
check_button: "🩺 Запустить проверки"
refresh: "🔄 Обновить"
to_list: "⬅️ К списку"
to_card: "⬅️ К контейнеру"
checking: "🩺 Проверяю контейнеры…"
report_title: "🩺 Проверки · без ошибок {ok} из {total} · {time} UTC"
report_container: "{light} {name}"
report_line: " {verdict} {title}{detail}"
check_running: "запущен"
check_health: "Docker health"
check_restarts: "рестарты за 5 мин не растут"
check_cpu: "CPU ниже порога"
check_mem: "RAM ниже порога"
check_log_errors: "нет ошибок в логе за 5 мин"
check_http: "HTTP-проба"
days: "д"
hours: "ч"
minutes: "мин"
seconds: "с"
+26 -1
View File
@@ -5,13 +5,22 @@ services:
env_file: .env
environment:
DATABASE_URL: postgresql+asyncpg://tgit:${POSTGRES_PASSWORD:-tgit}@postgres:5432/tgit
GIT_CLONE_DIR: /data/uploads
DOCKER_URL: http://docker-proxy:2375
volumes:
- uploads:/data
depends_on:
postgres:
condition: service_healthy
docker-proxy:
condition: service_started
restart: unless-stopped
mem_limit: 256m
mem_limit: 384m # ~190 MB idle; git clone/push and aiogram spikes need headroom
labels:
tgit.managed: "true"
# Shown on the «Статус» card; infra/deploy/remote.sh sets both, a local `make up` leaves them empty
tgit.sha: ${TGIT_SHA:-}
tgit.deployed_at: ${TGIT_DEPLOYED_AT:-}
logging:
options: { max-size: "10m", max-file: "3" }
@@ -35,5 +44,21 @@ services:
logging:
options: { max-size: "10m", max-file: "3" }
# The only holder of the Docker socket: GET on /containers/* and nothing else (ADR-006). The bot never sees the socket.
docker-proxy:
image: tecnativa/docker-socket-proxy:0.3.0
environment:
CONTAINERS: 1 # list, inspect, stats, logs
POST: 0 # no create/start/stop/exec — restarts go through tgit-runner (#13)
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
restart: unless-stopped
mem_limit: 64m
labels:
tgit.managed: "true"
logging:
options: { max-size: "10m", max-file: "3" }
volumes:
pgdata:
uploads:
+8 -5
View File
@@ -70,11 +70,11 @@ aiogram 3. Главное меню — reply-клавиатура (📝 Комм
### 2.4 `app/sourcecraft` — клиент API
`httpx.AsyncClient`, Bearer-токен, ретраи на сетевые ошибки и 5xx (экспоненциально, до 5 попыток),
pydantic-модели ответов. Методы: `list_runs`, `get_run`, `get_cube_logs`. Подробности — [`dev/SOURCECRAFT_API.md`](dev/SOURCECRAFT_API.md).
pydantic-модели ответов. Методы: `list_runs`, `get_run`, `get_cube_logs` (постранично), `run_commit_sha` — sha из лога куба `check/meta` (у прогона в API нет коммита, ADR-014). Подробности — [`dev/SOURCECRAFT_API.md`](dev/SOURCECRAFT_API.md).
### 2.5 `app/pipeline` — наблюдатель CI/CD
Фоновая задача. Пока есть незавершённые пуши — опрашивает `/cicd/runs` (10 с, бэкофф до 60 с), сопоставляет прогон с
пушем по sha. Отслеживает workflow `check` (CI) и `deploy` (CD) отдельно; на смене статуса — редактирует сообщение
пушем по sha (из куба `meta`, ADR-014). Отслеживает задачи `check` (CI) и `deploy` (CD) workflow `check-and-deploy` отдельно; на смене статуса — редактирует сообщение
о пуше и шлёт автору уведомление (успех / падение + лог упавшего куба). Падение `deploy` → событие для `alerts`.
Таймаут — 30 мин. Webhooks у SourceCraft нет (ADR-003).
@@ -84,8 +84,11 @@ pydantic-модели ответов. Методы: `list_runs`, `get_run`, `get
- **Карточка** — `inspect` (state, health, restart count, started_at, метки `tgit.sha`, `tgit.deployed_at`) + `stats`.
- **Проверки** (`runtime/checks.py`) — набор независимых функций `Check -> CheckResult(ok, detail)`:
`running`, `docker_health`, `restarts_stable` (сравнение с 5 мин назад, история из БД), `cpu_ok`, `mem_ok`,
`no_errors_in_logs` (5 мин, шаблон из `config/checks.yaml`), `http_probe` (если в `services/<name>/tgit.yaml`
объявлен `health_url`; проба из сети сервиса через runner). Результат — отчёт-таблица.
`no_errors_in_logs` (5 мин, шаблон из `config/checks.yaml`), `http_probe` (если у контейнера есть метка
`tgit.health_url`; пока нет runner (#13) — проба из сети бота). Каждая проверка — со своим таймаутом: упавшая или
зависшая даёт ❌ с причиной, остальные выполняются. Пороги и окно — в `config/checks.yaml`.
- **Изоляция** — прокси отдаёт контейнеры всего хоста, поэтому бот сам фильтрует по метке `tgit.managed=true` и
отказывает в `inspect`/`stats`/`logs` по id не из этого списка ещё до запроса.
- **Мониторинг** — поток `events` (`die`, `oom`, `health_status`, `restart`) и опрос `stats` раз в 15 с → события для `alerts`.
### 2.7 `app/logs` — grep-движок
@@ -138,7 +141,7 @@ SQLAlchemy 2 (async) + Alembic. PostgreSQL в Compose; в тестах — SQLit
| `drafts` | `id`, `user_id` UNIQUE, `base_sha`, `message`, `created_at`, `expires_at` |
| `draft_changes` | `id`, `draft_id`, `path`, `op` (`add`/`modify`/`delete`), `content` BYTEA |
| `pushes` | `id`, `user_id`, `sha`, `message`, `paths` JSON, `chat_id`, `message_id`, `ci_status`, `cd_status`, `created_at` |
| `pipeline_runs` | `id`, `push_id`, `workflow` (`check`/`deploy`), `run_slug`, `status`, `failed_cube`, `url`, `updated_at` |
| `pipeline_runs` | `id`, `push_id`, `stage` (`ci`/`cd` — задачи одного workflow `check-and-deploy`), `run_slug`, `status`, `failed_cube`, `url`, `updated_at` |
| `container_snapshots` | `container`, `at`, `restart_count`, `cpu_pct`, `mem_pct`, `health` — для проверок «за 5 мин» (TTL 1 сутки) |
| `alerts` | `id`, `kind`, `service`, `payload` JSON, `count`, `opened_at`, `last_at`, `resolved_at` |
| `alert_deliveries` | `alert_id`, `chat_id`, `message_id` |
+16
View File
@@ -121,3 +121,19 @@
**Решение:** конвейер из пяти шагов (SPEC §4.3.1): детерминированный фильтр → отдельный LLM-вызов «страж» с JSON-вердиктом `safe|attack` (невалидный ответ = `attack`) → отдельный LLM-вызов «планировщик», который получает только промт и список контейнеров и возвращает `LogQuery` → валидация → выполнение в Python. Пользователю показывается итоговый grep и найденные строки. Атака → отказ + алерт админам.
**Альтернативы:** один вызов LLM с инструкцией «откажись, если атака» — страж и исполнитель в одном контексте, injection обходит оба; передавать логи в LLM для поиска — injection из логов и утечка секретов.
**Последствия:** два LLM-вызова на запрос (латентность, токены → квота 30/сутки); даже при обходе стража вред ограничен схемой `LogQuery`.
## ADR-014: Связь «прогон ↔ коммит» — через куб `meta`, печатающий sha
**Дата:** 2026-10-14 · **Статус:** принято (уточняет ADR-003)
**Контекст:** бот должен понять, какой прогон CI относится к его пушу. В API SourceCraft у прогона нет sha коммита (ни в списке, ни в деталях), фильтра по коммиту нет; логи служебных кубов (`#git-clone`) API не отдаёт.
**Решение:** первый куб задачи `check` в `tgit-uploads` — `meta`: `echo "TGIT_COMMIT_SHA=$SOURCECRAFT_COMMIT_SHA"`. Бот читает его лог (`SourceCraftClient.run_commit_sha`) и сопоставляет прогон с пушем.
**Альтернативы:** запускать CI из бота вручную (`POST /cicd/runs` с `head.commit`) — номер прогона известен сразу, но push-триггер пришлось бы убрать, и ручные пуши в репозиторий остались бы без CI; сопоставление по времени — ломается при параллельных пушах.
**Последствия:** +1 короткий куб на прогон; куб `meta` нельзя удалять или переименовывать (он в `.sourcecraft/`, который бот защищает от правок); прогоны старше этого решения sha не имеют.
## ADR-015: CI/CD бота — на Gitea Actions (как в Vobot)
**Дата:** 2026-10-16 · **Статус:** принято (меняет ADR-010 в части CI/CD бота)
**Контекст:** SourceCraft отложен: ключ бота там не принимается, триггеры зависят от основной ветки, раннеры с очередью до минут. Гейт и деплой нужны сейчас.
**Решение:** CI (`.gitea/workflows/ci.yml`: ruff, mypy, миграции на Postgres, pytest) и CD (`.gitea/workflows/deploy.yml` → `infra/deploy/deploy.sh`) — на Gitea Actions, как у Vobot и meditron, на общих раннерах Gitea. Деплой — на хост Vobot, `dev`/`prod` по веткам, через пользователя `tgit-deploy`. Репозиторий загрузок на сервере — временно `HSE/tgit-uploads` в Gitea.
**Альтернативы:** ждать SourceCraft — блокирует проверку и демо.
**Последствия:** `.sourcecraft/ci.yaml` бота остаётся для будущего возврата; при возврате к SourceCraft меняются `GIT_UPLOADS_URL` и ключ, код не трогаем. На RF-сервере Telegram — только через прокси (`TELEGRAM_PROXY_URL`).
+45
View File
@@ -0,0 +1,45 @@
# Деплой бота
CD — `.gitea/workflows/deploy.yml` (Gitea Actions, ADR-015): push в `dev` → окружение **dev**, push в `main` → **prod**.
Под капотом — `infra/deploy/deploy.sh`, тот же скрипт работает руками и для любого Docker-хоста.
## Сервер (хост Vobot, 72.56.247.166)
| Что | Где |
|---|---|
| Пользователь | `tgit-deploy` (группа `docker`), ключ CI в `~/.ssh/authorized_keys` |
| Окружение | `/srv/tgit/<env>/` — `.env` (только здесь, 600), `app/` (текущий релиз), `app.old/` (предыдущий), `REVISION` |
| Контейнеры | Compose-проект `tgit-<env>`: `tgit-<env>-bot-1`, `tgit-<env>-postgres-1`; volumes `pgdata`, `uploads` |
| Telegram | с сервера напрямую недоступен → `TELEGRAM_PROXY_URL` (тот же прокси, что у meditron) |
| Репозиторий загрузок | временно `ssh://git@gitea.24studbank.ru:2222/HSE/tgit-uploads.git`, deploy key бота с записью |
Секреты репозитория в Gitea (Settings → Actions → Secrets): `DEPLOY_SSH_KEY` (приватный ключ `tgit-cd@gitea`), `DEPLOY_HOST` (`tgit-deploy@72.56.247.166`).
## Руками
```bash
DEPLOY_SSH_OPTS="-F /dev/null -i ~/.ssh/tgit_ci_deploy -o IdentitiesOnly=yes" infra/deploy/deploy.sh dev HEAD
```
Другой сервер: `DEPLOY_HOST=user@host`, на нём — Docker, пользователь в группе `docker`, `/srv/tgit/<env>/.env`.
## Проверить
```bash
ssh tgit-deploy@72.56.247.166 'cat /srv/tgit/dev/REVISION; docker ps --filter name=tgit-dev; docker logs --tail 20 tgit-dev-bot-1'
```
## Откат
Ошибка health при деплое печатает команду; руками:
```bash
ssh tgit-deploy@72.56.247.166
cd /srv/tgit/dev && rm -rf app && mv app.old app && cd app && docker compose -p tgit-dev up -d --build
```
## Завести prod
Нужен **отдельный токен бота** (один токен — один процесс polling): создать бота у @BotFather →
`/srv/tgit/prod/.env` по образцу dev (`TELEGRAM_BOT_TOKEN`, `ADMIN_IDS`, `POSTGRES_PASSWORD`, `GIT_UPLOADS_URL`,
`GIT_SSH_KEY_B64`, `TELEGRAM_PROXY_URL`, `LLM_PROVIDER`). Пока файла нет, деплой `main` пропускается.
+6
View File
@@ -79,6 +79,12 @@ curl -s -X PUT -H "$H" -H "$J" "$R/secrets/DEPLOY_HOST" -d "{\"value\":\"$(print
- Прогон: `slug` — числовой (`1000000`…), `status`: `created` → `processing` → `success`/`failed`; `event_type`:
`push`/`manual`/…. У кубов служебные шаги со `slug`, начинающимся на `#` (`#git-clone`, `#prepare-image-for-<cube>`).
- Если куб упал, остальные кубы задачи остаются в `created` и не запускаются.
- **В прогоне нет sha коммита** (ни в списке, ни в деталях), фильтра по коммиту нет → в `tgit-uploads` куб `check/meta` печатает `TGIT_COMMIT_SHA=…`, бот читает его лог (ADR-014).
- Логи служебных кубов (`#git-clone` и т.п.) API не отдаёт: `400 CITaskName.Pattern`.
- Логи постранично: `?page=N` с 1; читать по порядку, пока `page_complete: true`; следующая страница незаконченной — 404.
- Статусы прогона: `created`, `prepared`, `processing`, `success`, `failed`, `canceled`, `timeout`, `skipped`, `awaiting_approval`, `rejected`. Типы события: `push`, `pr_update`, `manual`, `restart`, `schedule`, `repository_event`.
- Очередь бесплатных раннеров бывает минутами: прогон может стоять в `prepared`/`created` до ~6 мин до старта.
- Веб-интерфейс — SPA, на любой путь отвечает 200: формат ссылки на прогон снаружи не проверить (`SOURCECRAFT_RUN_URL_TEMPLATE`).
- Код в кубе склонирован в `/sourcecraft/workspace` (`$SOURCECRAFT_WORKSPACE`) — это рабочий каталог. Переменные:
`SOURCECRAFT_COMMIT_SHA`, `SOURCECRAFT_COMMIT_REF_NAME` и др.
- Исходящих webhooks нет — статус CI получаем опросом (ADR-003).
+34
View File
@@ -0,0 +1,34 @@
#!/usr/bin/env bash
# Deploy a git ref of the bot to a server. Used by hand and by CD (.gitea/workflows/deploy.yml).
# infra/deploy/deploy.sh <dev|prod> [git-ref] default ref: HEAD
# Env: DEPLOY_HOST (default tgit-deploy@72.56.247.166 — the vobot host; any Docker host works)
# DEPLOY_SSH_OPTS (extra ssh/scp options, e.g. -i key)
# Server layout: /srv/tgit/<env>/{.env, app/, app.old/, REVISION}; secrets live only in /srv/tgit/<env>/.env.
# The host drops some ssh connections — every remote step is retried, and each one is safe to repeat.
set -euo pipefail
ENV=${1:?usage: deploy.sh <dev|prod> [ref]}
REF=${2:-HEAD}
case "$ENV" in dev | prod) ;; *) echo "env must be dev or prod" >&2; exit 2 ;; esac
HOST=${DEPLOY_HOST:-tgit-deploy@72.56.247.166}
OPTS=${DEPLOY_SSH_OPTS:-}
DIR=/srv/tgit/$ENV
SHA=$(git rev-parse --short "$REF")
TGZ=$(mktemp --suffix=.tgz)
trap 'rm -f "$TGZ"' EXIT
git archive --format=tar.gz -o "$TGZ" "$REF"
retry() { local n=0; until "$@"; do n=$((n + 1)); [ $n -ge 5 ] && return 1; echo "retry $n: $1" >&2; sleep $((n * 5)); done; }
# shellcheck disable=SC2086
if ! retry ssh $OPTS "$HOST" "test -f $DIR/.env"; then
echo "::notice::$HOST:$DIR/.env not found — environment '$ENV' is not configured, skipping deploy"
exit 0
fi
echo "deploy $REF ($SHA) -> $HOST:$DIR"
# shellcheck disable=SC2086
retry scp -q $OPTS "$TGZ" "$HOST:$DIR/release-$SHA.tgz"
# shellcheck disable=SC2086
retry scp -q $OPTS infra/deploy/remote.sh "$HOST:$DIR/remote.sh"
# shellcheck disable=SC2086
retry ssh $OPTS "$HOST" "bash $DIR/remote.sh $ENV $SHA"
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env bash
# Runs on the server: unpack release-<sha>.tgz, swap app/ atomically, compose up, wait for healthy.
# bash /srv/tgit/<env>/remote.sh <env> <sha> (uploaded by deploy.sh; safe to re-run)
set -euo pipefail
ENV=$1
SHA=$2
DIR=/srv/tgit/$ENV
PROJECT=tgit-$ENV
cd "$DIR"
if [ -f "release-$SHA.tgz" ]; then
rm -rf app.new && mkdir app.new && tar -xzf "release-$SHA.tgz" -C app.new && rm -f "release-$SHA.tgz"
cp .env app.new/.env
rm -rf app.old
[ -d app ] && mv app app.old
mv app.new app
fi
cd app
# Labels on the bot container, shown on its «Статус» card
export TGIT_SHA=$SHA TGIT_DEPLOYED_AT
TGIT_DEPLOYED_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)
docker compose -p "$PROJECT" up -d --build --remove-orphans </dev/null
status=""
for _ in $(seq 1 40); do
status=$(docker inspect -f '{{.State.Health.Status}}' "$PROJECT-bot-1" 2>/dev/null || true)
[ "$status" = healthy ] && break
sleep 3
done
if [ "$status" != healthy ]; then
echo "bot is '$status' after 2 min — last logs:" >&2
docker logs --tail 50 "$PROJECT-bot-1" >&2 || true
echo "rollback: cd $DIR && rm -rf app && mv app.old app && cd app && docker compose -p $PROJECT up -d --build" >&2
exit 1
fi
echo "$SHA" > ../REVISION
docker image prune -f >/dev/null
echo "bot healthy @ $SHA ($ENV)"
@@ -13,7 +13,13 @@ workflows:
tasks:
- name: check
cubes:
# The API doesn't tell which commit a run is for — the bot reads it from this log (ADR-014). Keep it first.
- name: meta
image: ghcr.io/astral-sh/uv:0.8-python3.12-bookworm-slim
script:
- echo "TGIT_COMMIT_SHA=$SOURCECRAFT_COMMIT_SHA"
- name: compile
needs: ["-"]
image: ghcr.io/astral-sh/uv:0.8-python3.12-bookworm-slim
script:
- python -m compileall -q -x '(^|/)\.' .
+5 -1
View File
@@ -14,6 +14,7 @@ dependencies = [
"pyyaml>=6.0",
"regex>=2024.7",
"segno>=1.6",
"aiohttp-socks>=0.9",
]
[dependency-groups]
@@ -26,6 +27,7 @@ dev = [
"pytest-cov>=5.0",
"respx>=0.21",
"aiosqlite>=0.20",
"types-regex>=2026.9.29.20261017",
]
[build-system]
@@ -42,6 +44,8 @@ extend-exclude = [".claude", "docs"]
[tool.ruff.lint]
select = ["E", "F", "I", "B", "UP", "SIM", "ASYNC", "RUF"]
# Russian UI texts, commit messages and comments are normal here, not homoglyph attacks
ignore = ["RUF001", "RUF002", "RUF003"]
[tool.mypy]
python_version = "3.12"
@@ -63,4 +67,4 @@ omit = ["app/__main__.py"]
show_missing = true
skip_covered = true
# Ratchet: raise as code lands, never lower. Target by M4 — 70 (SPEC §5.2).
fail_under = 45
fail_under = 80
View File
+138
View File
@@ -0,0 +1,138 @@
from datetime import UTC, datetime, timedelta
import pytest
from sqlalchemy import select
from app.access.service import AccessService, RedeemStatus, TgUser, hash_token
from app.config import AccessSettings
from app.db.models import Draft, FsmState, Invite, Role, User
from app.db.session import SessionMaker
ADMIN = 1
NEWBIE = TgUser(tg_id=500, full_name="Новичок", username="newbie")
class Clock:
def __init__(self) -> None:
self.now = datetime(2026, 10, 15, 12, 0, tzinfo=UTC)
def __call__(self) -> datetime:
return self.now
@pytest.fixture
def clock() -> Clock:
return Clock()
@pytest.fixture
async def svc(sessions: SessionMaker, clock: Clock) -> AccessService:
service = AccessService(sessions, AccessSettings(admin_ids=frozenset({ADMIN}), invite_ttl_hours=24), clock=clock)
await service.bootstrap_admins()
return service
async def test_bootstrap_is_idempotent_and_restores_admin(svc: AccessService, sessions: SessionMaker) -> None:
async with sessions.begin() as s:
admin = await s.get(User, ADMIN)
assert admin is not None
admin.is_active, admin.role = False, Role.MEMBER
await svc.bootstrap_admins()
await svc.bootstrap_admins()
member = await svc.get_member(ADMIN)
assert member is not None and member.role is Role.ADMIN
async def test_invite_stores_only_hash(svc: AccessService, sessions: SessionMaker, clock: Clock) -> None:
invite = await svc.create_invite(ADMIN)
assert len(invite.token) == 22 and invite.token.replace("-", "").replace("_", "").isalnum()
assert invite.expires_at == clock.now + timedelta(hours=24)
async with sessions() as s:
stored = await s.scalar(select(Invite))
assert stored is not None and stored.token_hash == hash_token(invite.token) != invite.token
async def test_redeem_once_then_rejected_for_others(svc: AccessService, sessions: SessionMaker) -> None:
invite = await svc.create_invite(ADMIN)
assert await svc.redeem(invite.token, NEWBIE) is RedeemStatus.OK
assert await svc.redeem(invite.token, NEWBIE) is RedeemStatus.ALREADY_MEMBER
assert await svc.redeem(invite.token, TgUser(tg_id=501, full_name="x")) is RedeemStatus.REJECTED
member = await svc.get_member(NEWBIE.tg_id)
assert member is not None and member.role is Role.MEMBER and member.invited_by == ADMIN
async with sessions() as s:
stored = await s.scalar(select(Invite))
assert stored is not None and stored.used_by == NEWBIE.tg_id
assert await s.get(User, 501) is None # the loser's user row was rolled back
async def test_expired_invite_rejected(svc: AccessService, clock: Clock) -> None:
invite = await svc.create_invite(ADMIN)
clock.now += timedelta(hours=24, seconds=1)
assert await svc.redeem(invite.token, NEWBIE) is RedeemStatus.REJECTED
assert await svc.get_member(NEWBIE.tg_id) is None
async def test_brute_force_is_throttled_even_with_a_valid_token(svc: AccessService, clock: Clock) -> None:
for i in range(5):
assert await svc.redeem(f"guess-{i}", NEWBIE) is RedeemStatus.REJECTED
invite = await svc.create_invite(ADMIN)
assert await svc.redeem(invite.token, NEWBIE) is RedeemStatus.THROTTLED
clock.now += timedelta(hours=1, seconds=1)
assert await svc.redeem(invite.token, NEWBIE) is RedeemStatus.OK
async def test_revoke_drops_access_draft_and_dialog(svc: AccessService, sessions: SessionMaker, clock: Clock) -> None:
await svc.redeem((await svc.create_invite(ADMIN)).token, NEWBIE)
async with sessions.begin() as s:
s.add(Draft(user_id=NEWBIE.tg_id, base_sha="0" * 40, expires_at=clock.now + timedelta(hours=1)))
s.add(FsmState(key=f"42:{NEWBIE.tg_id}:{NEWBIE.tg_id}:::default", state="x", data={}))
s.add(FsmState(key=f"42:{ADMIN}:{ADMIN}:::default", state="y", data={}))
assert await svc.revoke(NEWBIE.tg_id)
assert await svc.get_member(NEWBIE.tg_id) is None
async with sessions() as s:
assert await s.scalar(select(Draft)) is None
assert [r.key for r in await s.scalars(select(FsmState))] == [f"42:{ADMIN}:{ADMIN}:::default"]
async def test_admin_cannot_be_revoked_and_unknown_is_noop(svc: AccessService) -> None:
assert not await svc.revoke(ADMIN)
assert not await svc.revoke(999)
async def test_revoked_member_can_be_invited_again(svc: AccessService) -> None:
await svc.redeem((await svc.create_invite(ADMIN)).token, NEWBIE)
await svc.revoke(NEWBIE.tg_id)
assert await svc.redeem((await svc.create_invite(ADMIN)).token, NEWBIE) is RedeemStatus.OK
assert await svc.get_member(NEWBIE.tg_id) is not None
async def test_members_listed_admins_first(svc: AccessService) -> None:
await svc.redeem((await svc.create_invite(ADMIN)).token, NEWBIE)
assert [m.tg_id for m in await svc.list_members()] == [ADMIN, NEWBIE.tg_id]
async def test_touch_is_throttled_but_tracks_renames(svc: AccessService, clock: Clock) -> None:
member = await svc.get_member(ADMIN)
assert member is not None
await svc.touch(member, TgUser(ADMIN, "Админ", "boss"))
first = await svc.get_member(ADMIN)
assert first is not None and first.full_name == "Админ" and first.last_seen_at is not None
clock.now += timedelta(minutes=1)
await svc.touch(first, TgUser(ADMIN, "Админ", "boss"))
same = await svc.get_member(ADMIN)
assert same is not None and same.last_seen_at == first.last_seen_at # no write within 5 minutes
await svc.touch(same, TgUser(ADMIN, "Новое имя", "boss"))
renamed = await svc.get_member(ADMIN)
assert renamed is not None and renamed.full_name == "Новое имя"
+101
View File
@@ -0,0 +1,101 @@
import pytest
from aiogram.methods import AnswerCallbackQuery, EditMessageText, SendDocument, SendMessage
from aiogram.types import InlineKeyboardMarkup
from app.access.service import AccessService
from app.bot.routers.commit import Act, CommitCb, human_size
from app.db.session import SessionMaker
from app.main import build_dispatcher
from app.repo.git import UploadsRepo
from app.texts import Texts
from tests.conftest import ADMIN
from tests.fakes import TgClient
@pytest.fixture
def tg(texts: Texts, sessions: SessionMaker, access: AccessService, repo: UploadsRepo) -> TgClient:
return TgClient(build_dispatcher(texts, sessions, access, repo))
def labels(markup: object) -> list[str]:
assert isinstance(markup, InlineKeyboardMarkup)
return [b.text for row in markup.inline_keyboard for b in row]
def cb(action: Act, i: int = 0) -> str:
return CommitCb(a=action, i=i).pack()
async def test_root_listing(tg: TgClient, texts: Texts) -> None:
await tg.send(ADMIN, texts.menu.commit)
[reply] = tg.session.sent(SendMessage)
assert reply.text == texts.commit.title.format(path="")
got = labels(reply.reply_markup)
assert got[:4] == ["📁 docs", "📁 many", "📁 services", "📄 README.md · 10 Б"]
assert texts.commit.up not in got # already at the root
async def test_open_dir_then_file_card_then_back(tg: TgClient, texts: Texts) -> None:
await tg.send(ADMIN, texts.menu.commit)
await tg.press(ADMIN, cb(Act.OPEN, 2)) # services
await tg.press(ADMIN, cb(Act.OPEN, 0)) # services/hello
await tg.press(ADMIN, cb(Act.OPEN, 0)) # services/hello/main.py
hello_dir, hello_sub, card = tg.session.sent(EditMessageText)
assert hello_dir.text == texts.commit.title.format(path="services")
assert texts.commit.up in labels(hello_dir.reply_markup)
assert hello_sub.text == texts.commit.title.format(path="services/hello")
assert card.text.startswith("📄 /services/hello/main.py") and "init" in card.text
assert labels(card.reply_markup) == [
texts.commit.view,
texts.commit.replace,
texts.commit.delete,
texts.commit.to_dir,
]
await tg.press(ADMIN, cb(Act.TO_DIR))
await tg.press(ADMIN, cb(Act.UP))
*_, back_to_dir, up = tg.session.sent(EditMessageText)
assert back_to_dir.text == texts.commit.title.format(path="services/hello")
assert up.text == texts.commit.title.format(path="services")
async def test_view_text_inline_and_binary_as_document(tg: TgClient, texts: Texts) -> None:
await tg.send(ADMIN, texts.menu.commit)
await tg.press(ADMIN, cb(Act.OPEN, 0)) # docs
await tg.press(ADMIN, cb(Act.OPEN, 1)) # мои заметки.md
await tg.press(ADMIN, cb(Act.VIEW))
await tg.press(ADMIN, cb(Act.TO_DIR))
await tg.press(ADMIN, cb(Act.OPEN, 0)) # logo.bin
await tg.press(ADMIN, cb(Act.VIEW))
inline = [m for m in tg.session.sent(SendMessage) if m.parse_mode == "HTML"]
assert len(inline) == 1 and "<pre>привет\n</pre>" in inline[0].text
[doc] = tg.session.sent(SendDocument)
assert doc.caption == texts.commit.sent_as_file
async def test_pagination(tg: TgClient, texts: Texts) -> None:
await tg.send(ADMIN, texts.menu.commit)
await tg.press(ADMIN, cb(Act.OPEN, 1)) # many: 25 files
await tg.press(ADMIN, cb(Act.PAGE, 1))
first, second = tg.session.sent(EditMessageText)
assert texts.commit.page.format(page=1, pages=2) in first.text
assert labels(first.reply_markup)[0].startswith("📄 f00.txt") and texts.commit.next in labels(first.reply_markup)
assert labels(second.reply_markup)[0].startswith("📄 f20.txt") and texts.commit.prev in labels(second.reply_markup)
async def test_stale_button_is_ignored(tg: TgClient, texts: Texts) -> None:
await tg.send(ADMIN, texts.menu.commit)
await tg.press(ADMIN, cb(Act.OPEN, 99))
[stale] = tg.session.sent(AnswerCallbackQuery)
assert stale.text is None
assert tg.session.sent(EditMessageText) == []
@pytest.mark.parametrize(("n", "text"), [(10, "10 Б"), (2048, "2.0 КБ"), (5 * 1024 * 1024, "5.0 МБ")])
def test_human_size(n: int, text: str) -> None:
assert human_size(n) == text
+162
View File
@@ -0,0 +1,162 @@
import pytest
from aiogram.methods import AnswerCallbackQuery, EditMessageText, SendMessage
from aiogram.types import InlineKeyboardMarkup
from app.access.service import AccessService
from app.bot.routers.commit import Act, CommitCb
from app.db.session import SessionMaker
from app.main import build_dispatcher
from app.repo.git import UploadsRepo
from app.texts import Texts
from tests.conftest import ADMIN
from tests.fakes import TgClient
@pytest.fixture
def tg(texts: Texts, sessions: SessionMaker, access: AccessService, repo: UploadsRepo) -> TgClient:
return TgClient(build_dispatcher(texts, sessions, access, repo))
def cb(action: Act, i: int = 0) -> str:
return CommitCb(a=action, i=i).pack()
def labels(markup: object) -> list[str]:
assert isinstance(markup, InlineKeyboardMarkup)
return [b.text for row in markup.inline_keyboard for b in row]
def last_text(tg: TgClient) -> SendMessage:
return tg.session.sent(SendMessage)[-1]
async def to_hello(tg: TgClient, texts: Texts) -> None:
await tg.send(ADMIN, texts.menu.commit)
await tg.press(ADMIN, cb(Act.OPEN, 2)) # services
await tg.press(ADMIN, cb(Act.OPEN, 0)) # services/hello
async def test_new_file_from_fenced_text(tg: TgClient, texts: Texts) -> None:
await to_hello(tg, texts)
await tg.press(ADMIN, cb(Act.NEW_FILE))
await tg.send(ADMIN, "utils.py")
await tg.send(ADMIN, "```python\ndef add(a, b):\n return a + b\n```")
reply = last_text(tg)
assert reply.text.startswith(texts.commit.added.format(path="services/hello/utils.py"))
got = labels(reply.reply_markup)
assert "🆕 📄 utils.py · 31 Б" in got # fence stripped, no trailing newline
assert texts.commit.draft_button.format(count=1) in got
await tg.press(ADMIN, cb(Act.OPEN, 1)) # utils.py (main.py is 0)
await tg.press(ADMIN, cb(Act.VIEW))
view = last_text(tg)
assert "def add(a, b):" in view.text and "```" not in view.text
async def test_replace_with_document_shows_diff(tg: TgClient, texts: Texts) -> None:
await to_hello(tg, texts)
await tg.press(ADMIN, cb(Act.OPEN, 0)) # main.py
await tg.press(ADMIN, cb(Act.REPLACE))
await tg.send_document(ADMIN, "main.py", b"print('hello')\nprint('bye')\n")
reply = last_text(tg)
assert reply.text.startswith(texts.commit.modified.format(path="services/hello/main.py", added=1, removed=0))
assert "✏️ 📄 main.py · 28 Б" in labels(reply.reply_markup)
async def test_delete_marks_and_draft_lists(tg: TgClient, texts: Texts) -> None:
await tg.send(ADMIN, texts.menu.commit)
await tg.press(ADMIN, cb(Act.OPEN, 3)) # README.md
await tg.press(ADMIN, cb(Act.DELETE))
deleted = tg.session.sent(EditMessageText)[-1]
assert deleted.text.startswith(texts.commit.deleted.format(path="README.md"))
assert "🗑 📄 README.md · 10 Б" in labels(deleted.reply_markup)
await tg.press(ADMIN, cb(Act.DRAFT))
draft = tg.session.sent(EditMessageText)[-1]
assert "D /README.md" in draft.text
await tg.press(ADMIN, cb(Act.UNSTAGE, 0))
assert tg.session.sent(EditMessageText)[-1].text == texts.commit.draft_empty
async def test_syntax_error_force_or_fix(tg: TgClient, texts: Texts) -> None:
await to_hello(tg, texts)
await tg.press(ADMIN, cb(Act.NEW_FILE))
await tg.send(ADMIN, "bad.py")
await tg.send(ADMIN, "def f(:\n pass")
warn = last_text(tg)
assert warn.text.startswith("⚠️") and "1" in warn.text
assert labels(warn.reply_markup) == [texts.commit.fix, texts.commit.force]
await tg.press(ADMIN, cb(Act.FIX))
assert tg.session.sent(EditMessageText)[-1].text == texts.commit.ask_content.format(path="services/hello/bad.py")
await tg.send(ADMIN, "def f(:\n pass")
await tg.press(ADMIN, cb(Act.FORCE))
assert last_text(tg).text.startswith(texts.commit.added.format(path="services/hello/bad.py"))
async def test_virtual_folder_becomes_real_with_a_file(tg: TgClient, texts: Texts) -> None:
await tg.send(ADMIN, texts.menu.commit)
await tg.press(ADMIN, cb(Act.OPEN, 2)) # services
await tg.press(ADMIN, cb(Act.NEW_DIR))
await tg.send(ADMIN, "api")
inside = last_text(tg)
assert inside.text == f"{texts.commit.title.format(path='services/api')}\n{texts.commit.empty}"
await tg.press(ADMIN, cb(Act.NEW_FILE))
await tg.send(ADMIN, "main.py")
await tg.send(ADMIN, "print('api')")
await tg.press(ADMIN, cb(Act.UP))
services = tg.session.sent(EditMessageText)[-1]
assert "🆕 📁 api" in labels(services.reply_markup)
async def test_names_rejected(tg: TgClient, texts: Texts) -> None:
await tg.send(ADMIN, texts.menu.commit)
await tg.press(ADMIN, cb(Act.NEW_FILE))
for bad in ["../etc", "README.md", "ruff.toml"]:
await tg.send(ADMIN, bad)
assert [m.text for m in tg.session.sent(SendMessage)][-3:] == [
texts.commit.bad_name,
texts.commit.exists,
texts.commit.protected,
]
async def test_binary_document_rejected(tg: TgClient, texts: Texts) -> None:
await to_hello(tg, texts)
await tg.press(ADMIN, cb(Act.NEW_FILE))
await tg.send(ADMIN, "img.png")
await tg.send_document(ADMIN, "img.png", b"\x89PNG\x00\x00")
assert last_text(tg).text == texts.commit.binary
async def test_draft_survives_a_restart(
texts: Texts, sessions: SessionMaker, access: AccessService, repo: UploadsRepo
) -> None:
first = TgClient(build_dispatcher(texts, sessions, access, repo))
await first.send(ADMIN, texts.menu.commit)
await first.press(ADMIN, cb(Act.OPEN, 3))
await first.press(ADMIN, cb(Act.DELETE))
second = TgClient(build_dispatcher(texts, sessions, access, repo)) # same DB, new bot process
await second.send(ADMIN, texts.menu.commit)
assert texts.commit.draft_button.format(count=1) in labels(second.session.sent(SendMessage)[-1].reply_markup)
async def test_protected_file_cannot_be_replaced(tg: TgClient, texts: Texts, repo: UploadsRepo) -> None:
await tg.send(ADMIN, texts.menu.commit)
# fake a card for a protected path via FSM: open README then rewrite the selection
await tg.press(ADMIN, cb(Act.OPEN, 3))
state = tg.dp.fsm.get_context(tg.bot, chat_id=ADMIN, user_id=ADMIN)
await state.update_data(commit_file=".sourcecraft/ci.yaml")
await tg.press(ADMIN, cb(Act.REPLACE))
answer = tg.session.sent(AnswerCallbackQuery)[-1]
assert answer.text == texts.commit.protected and answer.show_alert
+92
View File
@@ -0,0 +1,92 @@
from pathlib import Path
import pytest
from aiogram.methods import EditMessageText, SendMessage
from aiogram.types import InlineKeyboardMarkup
from app.access.service import AccessService
from app.bot.routers.commit import Act, CommitCb
from app.db.session import SessionMaker
from app.main import build_dispatcher
from app.repo.git import UploadsRepo
from app.texts import Texts
from tests.conftest import ADMIN, git
from tests.fakes import TgClient
@pytest.fixture
def tg(texts: Texts, sessions: SessionMaker, access: AccessService, repo: UploadsRepo) -> TgClient:
return TgClient(build_dispatcher(texts, sessions, access, repo, "https://example.test/commit/{sha}"))
def cb(action: Act, i: int = 0) -> str:
return CommitCb(a=action, i=i).pack()
def labels(markup: object) -> list[str]:
assert isinstance(markup, InlineKeyboardMarkup)
return [b.text for row in markup.inline_keyboard for b in row]
async def test_message_then_push_lands_in_remote(tg: TgClient, texts: Texts, remote: Path, tmp_path: Path) -> None:
t = texts.commit
await tg.send(ADMIN, texts.menu.commit)
await tg.press(ADMIN, cb(Act.NEW_FILE))
await tg.send(ADMIN, "notes.txt")
await tg.send(ADMIN, "привет из телеграма")
await tg.press(ADMIN, cb(Act.DRAFT))
draft = tg.session.sent(EditMessageText)[-1]
assert t.message_missing in draft.text and t.push_button not in labels(draft.reply_markup)
await tg.press(ADMIN, cb(Act.MESSAGE))
await tg.send(ADMIN, "x" * 80) # first line too long
assert tg.session.sent(SendMessage)[-1].text == t.bad_message
await tg.send(ADMIN, "docs: заметки")
with_message = tg.session.sent(SendMessage)[-1]
assert t.message_current.format(message="docs: заметки") in with_message.text
assert t.push_button in labels(with_message.reply_markup)
await tg.press(ADMIN, cb(Act.PUSH))
assert tg.session.sent(EditMessageText)[-1].text.startswith("Запушить в main?")
await tg.press(ADMIN, cb(Act.PUSH_GO))
pushing, done = tg.session.sent(EditMessageText)[-2:]
assert pushing.text == t.pushing
assert done.text.startswith("✅ Запушено: ") and "docs: заметки" in done.text
assert "https://example.test/commit/" in done.text and done.text.endswith(t.ci_pending)
clone = tmp_path / "verify"
git(tmp_path, "clone", "-q", str(remote), str(clone))
assert (clone / "notes.txt").read_text(encoding="utf-8") == "привет из телеграма"
# the draft is gone after a push
tg.session.requests.clear()
await tg.send(ADMIN, texts.menu.commit)
assert not any(label.startswith("🧾") for label in labels(tg.session.sent(SendMessage)[-1].reply_markup))
async def test_commit_button_in_folder_asks_message_then_confirms(
tg: TgClient, texts: Texts, remote: Path, tmp_path: Path
) -> None:
t = texts.commit
await tg.send(ADMIN, texts.menu.commit)
await tg.press(ADMIN, cb(Act.NEW_FILE))
await tg.send(ADMIN, "quick.txt")
await tg.send(ADMIN, "быстро")
folder = tg.session.sent(SendMessage)[-1]
assert t.commit_button.format(count=1) in labels(folder.reply_markup)
await tg.press(ADMIN, cb(Act.COMMIT))
assert tg.session.sent(EditMessageText)[-1].text == t.ask_message
await tg.send(ADMIN, "feat: быстрый коммит")
confirm = tg.session.sent(SendMessage)[-1]
assert confirm.text.startswith("Запушить в main?") and "feat: быстрый коммит" in confirm.text
assert labels(confirm.reply_markup) == [t.push_go, t.edit_message, texts.menu.back]
await tg.press(ADMIN, cb(Act.PUSH_GO))
assert tg.session.sent(EditMessageText)[-1].text.startswith("✅ Запушено: ")
clone = tmp_path / "verify2"
git(tmp_path, "clone", "-q", str(remote), str(clone))
assert (clone / "quick.txt").read_text(encoding="utf-8") == "быстро"
+6 -4
View File
@@ -2,17 +2,19 @@ import pytest
from aiogram.methods import AnswerCallbackQuery, EditMessageText, SendMessage
from aiogram.types import InlineKeyboardMarkup, ReplyKeyboardMarkup
from app.access.service import AccessService
from app.bot.menu import Nav, NavAction
from app.config import Settings
from app.db.session import SessionMaker
from app.main import build_dispatcher
from app.repo.git import UploadsRepo
from app.texts import Texts
from tests.conftest import ADMIN, OUTSIDER
from tests.fakes import TgClient
@pytest.fixture
def tg(settings: Settings, texts: Texts) -> TgClient:
return TgClient(build_dispatcher(settings, texts))
def tg(texts: Texts, sessions: SessionMaker, access: AccessService, repo: UploadsRepo) -> TgClient:
return TgClient(build_dispatcher(texts, sessions, access, repo))
def buttons(markup: ReplyKeyboardMarkup) -> list[str]:
@@ -44,7 +46,7 @@ async def test_outsider_stale_button_is_swallowed(tg: TgClient) -> None:
assert len(tg.session.sent(AnswerCallbackQuery)) == 1
@pytest.mark.parametrize("section", ["commit", "status", "logs", "alerts", "members"])
@pytest.mark.parametrize("section", ["logs", "alerts"])
async def test_menu_button_opens_section_with_nav(tg: TgClient, texts: Texts, section: str) -> None:
await tg.send(ADMIN, getattr(texts.menu, section))
+48
View File
@@ -0,0 +1,48 @@
from aiogram.fsm.state import State, StatesGroup
from aiogram.fsm.storage.base import StorageKey
from sqlalchemy import func, select
from app.bot.fsm_storage import SqlStorage, key_to_str
from app.db.models import FsmState
from app.db.session import SessionMaker
KEY = StorageKey(bot_id=42, chat_id=7, user_id=7)
class Flow(StatesGroup):
name = State()
async def test_state_and_data_survive_a_new_storage(sessions: SessionMaker) -> None:
first = SqlStorage(sessions)
await first.set_state(KEY, Flow.name)
await first.set_data(KEY, {"path": "services/a", "page": 2})
# A fresh storage on the same DB = the bot after a restart
second = SqlStorage(sessions)
assert await second.get_state(KEY) == Flow.name.state
assert await second.get_data(KEY) == {"path": "services/a", "page": 2}
async def test_unknown_key_is_empty(sessions: SessionMaker) -> None:
storage = SqlStorage(sessions)
assert await storage.get_state(KEY) is None
assert await storage.get_data(KEY) == {}
async def test_cleared_state_removes_row(sessions: SessionMaker) -> None:
storage = SqlStorage(sessions)
await storage.set_state(KEY, "x")
await storage.set_data(KEY, {"a": 1})
await storage.set_state(KEY, None)
await storage.set_data(KEY, {})
await storage.close()
async with sessions() as s:
assert await s.scalar(select(func.count()).select_from(FsmState)) == 0
def test_key_includes_every_part() -> None:
key = StorageKey(bot_id=1, chat_id=2, user_id=3, thread_id=4, business_connection_id="b", destiny="d")
assert key_to_str(key) == "1:2:3:4:b:d"
assert key_to_str(KEY) == "42:7:7:::default"
+95
View File
@@ -0,0 +1,95 @@
import re
import pytest
from aiogram.methods import AnswerCallbackQuery, EditMessageText, SendMessage, SendPhoto
from aiogram.types import ReplyKeyboardMarkup
from app.access.service import AccessService
from app.bot.routers.members import MembersAction, MembersCb
from app.db.session import SessionMaker
from app.main import build_dispatcher
from app.repo.git import UploadsRepo
from app.texts import Texts
from tests.conftest import ADMIN
from tests.fakes import TgClient
NEWBIE = 3003
@pytest.fixture
def tg(texts: Texts, sessions: SessionMaker, access: AccessService, repo: UploadsRepo) -> TgClient:
return TgClient(build_dispatcher(texts, sessions, access, repo))
async def invite_token(tg: TgClient) -> str:
await tg.press(ADMIN, MembersCb(action=MembersAction.INVITE).pack())
[photo] = tg.session.sent(SendPhoto)
match = re.search(r"https://t\.me/tgit_test_bot\?start=([A-Za-z0-9_-]+)", photo.caption or "")
assert match, photo.caption
tg.session.requests.clear()
return match.group(1)
async def test_admin_sees_members_list(tg: TgClient, texts: Texts) -> None:
await tg.send(ADMIN, texts.menu.members)
[reply] = tg.session.sent(SendMessage)
assert reply.text.startswith(texts.members.title.format(count=1))
async def test_full_invite_flow(tg: TgClient, texts: Texts) -> None:
token = await invite_token(tg)
await tg.send(NEWBIE, f"/start {token}")
[welcome] = tg.session.sent(SendMessage)
assert welcome.text == texts.start.welcome
assert isinstance(welcome.reply_markup, ReplyKeyboardMarkup)
labels = [b.text for row in welcome.reply_markup.keyboard for b in row]
assert texts.menu.members not in labels # a member, not an admin
# the same QR again, from someone else
tg.session.requests.clear()
await tg.send(NEWBIE + 1, f"/start {token}")
assert [r.text for r in tg.session.sent(SendMessage)] == [texts.start.invite_rejected]
async def test_bad_token_rejected_and_bare_start_refused(tg: TgClient, texts: Texts) -> None:
await tg.send(NEWBIE, "/start nope")
await tg.send(NEWBIE + 1, "/start")
assert [r.text for r in tg.session.sent(SendMessage)] == [texts.start.invite_rejected, texts.start.no_access]
async def test_revoke_with_confirmation_cuts_access(tg: TgClient, texts: Texts) -> None:
await tg.send(NEWBIE, f"/start {await invite_token(tg)}")
tg.session.requests.clear()
await tg.press(ADMIN, MembersCb(action=MembersAction.ASK_REVOKE, tg_id=NEWBIE).pack())
await tg.press(ADMIN, MembersCb(action=MembersAction.REVOKE, tg_id=NEWBIE).pack())
confirm, done = tg.session.sent(EditMessageText)
assert confirm.text == texts.members.confirm.format(name=f"u{NEWBIE}")
assert done.text.startswith(texts.members.revoked.format(name=f"u{NEWBIE}"))
tg.session.requests.clear()
await tg.send(NEWBIE, texts.menu.commit)
assert [r.text for r in tg.session.sent(SendMessage)] == [texts.start.no_access]
async def test_member_cannot_use_admin_actions(tg: TgClient, texts: Texts) -> None:
await tg.send(NEWBIE, f"/start {await invite_token(tg)}")
tg.session.requests.clear()
await tg.send(NEWBIE, texts.menu.members)
await tg.press(NEWBIE, MembersCb(action=MembersAction.INVITE).pack())
assert tg.session.sent(SendMessage) == [] and tg.session.sent(SendPhoto) == []
assert len(tg.session.sent(AnswerCallbackQuery)) == 1
async def test_cancel_revoke_returns_to_list(tg: TgClient, texts: Texts) -> None:
await tg.press(ADMIN, MembersCb(action=MembersAction.LIST).pack())
[edit] = tg.session.sent(EditMessageText)
assert edit.text.startswith(texts.members.title.format(count=1))
+143
View File
@@ -0,0 +1,143 @@
from collections.abc import AsyncIterator
import httpx
import pytest
from aiogram.methods import AnswerCallbackQuery, EditMessageText, SendMessage
from app.access.service import AccessService
from app.bot.routers.status import StatusAct, StatusCb, duration
from app.db.session import SessionMaker
from app.main import build_dispatcher
from app.repo.git import UploadsRepo
from app.runtime.checks import Checker, load_checks
from app.runtime.docker import DockerClient
from app.runtime.snapshots import SnapshotStore
from app.texts import Texts
from tests.conftest import ADMIN, ROOT
from tests.fakes import TgClient
from tests.runtime.fake_docker import FakeDocker
@pytest.fixture
def fake() -> FakeDocker:
fake = FakeDocker()
fake.add("tgit-dev-bot-1", service="bot", labels={"tgit.sha": "22d9f0c5abcdef0123"})
fake.add("unhealthy", service="unhealthy", health="unhealthy")
fake.add("vobot-api-1", managed=False)
return fake
@pytest.fixture
async def docker(fake: FakeDocker) -> AsyncIterator[DockerClient]:
async with fake.client() as client:
yield client
@pytest.fixture
def tg(
texts: Texts, sessions: SessionMaker, access: AccessService, repo: UploadsRepo, docker: DockerClient
) -> TgClient:
checker = Checker(
docker,
SnapshotStore(sessions),
load_checks(ROOT / "config"),
httpx.AsyncClient(transport=httpx.MockTransport(lambda r: httpx.Response(200))),
)
return TgClient(build_dispatcher(texts, sessions, access, repo, docker=docker, checker=checker))
def short(fake: FakeDocker, name: str) -> str:
return next(c.id[:12] for c in fake.containers.values() if c.name == name)
async def test_list_shows_managed_containers_with_lights(tg: TgClient, texts: Texts) -> None:
await tg.send(ADMIN, texts.menu.status)
[reply] = tg.session.sent(SendMessage)
assert "🟢 tgit-dev-bot-1 · running, healthy" in reply.text
assert "🔴 unhealthy · running, unhealthy" in reply.text
assert "vobot" not in reply.text
labels = [b.text for row in reply.reply_markup.inline_keyboard for b in row]
assert texts.status.check_button in labels and texts.status.refresh in labels
async def test_card(tg: TgClient, texts: Texts, fake: FakeDocker) -> None:
await tg.send(ADMIN, texts.menu.status)
await tg.press(ADMIN, StatusCb(a=StatusAct.CARD, id=short(fake, "tgit-dev-bot-1")).pack())
[card] = tg.session.sent(EditMessageText)
assert card.text.startswith("🟢 tgit-dev-bot-1")
assert "Ревизия: 22d9f0c5abcd" in card.text
assert "CPU: 5.0% · RAM: 64.0 МБ из 256.0 МБ (25%)" in card.text
assert "рестартов: 0" in card.text
async def test_card_of_stopped_container(tg: TgClient, texts: Texts, fake: FakeDocker) -> None:
fake.add("stopped", state="exited", health=None, exit_code=1)
await tg.press(ADMIN, StatusCb(a=StatusAct.CARD, id=short(fake, "stopped")).pack())
[card] = tg.session.sent(EditMessageText)
assert card.text.startswith("🔴 stopped")
assert "exited (exit 1)" in card.text
assert f"CPU: {texts.status.none}" in card.text
async def test_run_all_checks(tg: TgClient, texts: Texts) -> None:
await tg.press(ADMIN, StatusCb(a=StatusAct.CHECK).pack())
[report] = tg.session.sent(EditMessageText)
assert report.text.startswith("🩺 Проверки · без ошибок 1 из 2")
assert "✅ tgit-dev-bot-1" in report.text
assert "❌ unhealthy" in report.text
assert f"❌ {texts.status.check_health} · unhealthy" in report.text
[ack, *_] = tg.session.sent(AnswerCallbackQuery)
assert ack.text == texts.status.checking
async def test_checks_of_one_container_offer_way_back(tg: TgClient, texts: Texts, fake: FakeDocker) -> None:
await tg.press(ADMIN, StatusCb(a=StatusAct.CHECK, id=short(fake, "unhealthy")).pack())
[report] = tg.session.sent(EditMessageText)
assert "без ошибок 0 из 1" in report.text
labels = [b.text for row in report.reply_markup.inline_keyboard for b in row]
assert texts.status.to_card in labels
async def test_foreign_or_vanished_container_is_refused(tg: TgClient, texts: Texts, fake: FakeDocker) -> None:
await tg.press(ADMIN, StatusCb(a=StatusAct.CARD, id=short(fake, "vobot-api-1")).pack())
assert not tg.session.sent(EditMessageText)
[alert] = tg.session.sent(AnswerCallbackQuery)
assert alert.text == texts.status.unavailable and alert.show_alert
async def test_docker_down(tg: TgClient, texts: Texts, fake: FakeDocker) -> None:
fake.down = True
await tg.send(ADMIN, texts.menu.status)
assert [r.text for r in tg.session.sent(SendMessage)] == [texts.status.unavailable]
async def test_without_docker_configured(
texts: Texts, sessions: SessionMaker, access: AccessService, repo: UploadsRepo
) -> None:
tg = TgClient(build_dispatcher(texts, sessions, access, repo))
await tg.send(ADMIN, texts.menu.status)
await tg.press(ADMIN, StatusCb(a=StatusAct.CHECK).pack())
assert [r.text for r in tg.session.sent(SendMessage)] == [texts.status.unavailable]
assert tg.session.sent(AnswerCallbackQuery)[0].text == texts.status.unavailable
async def test_refresh_list(tg: TgClient, texts: Texts) -> None:
await tg.press(ADMIN, StatusCb(a=StatusAct.LIST).pack())
[edit] = tg.session.sent(EditMessageText)
assert edit.text.startswith("🩺 Статус · 2 конт.")
@pytest.mark.parametrize(
("seconds", "expected"), [(5, "5 с"), (125, "2 мин"), (3 * 3600 + 120, "3 ч 2 мин"), (2 * 86400 + 3600, "2 д 1 ч")]
)
def test_duration(texts: Texts, seconds: float, expected: str) -> None:
assert duration(texts.status, seconds) == expected
+70 -1
View File
@@ -1,8 +1,15 @@
import subprocess
from collections.abc import AsyncIterator
from pathlib import Path
import pytest
from sqlalchemy.ext.asyncio import AsyncEngine
from app.config import AccessSettings, RuntimeSettings, Settings, TelegramSettings
from app.access.service import AccessService
from app.config import AccessSettings, GitSettings, RuntimeSettings, Settings, TelegramSettings
from app.db.migrate import upgrade
from app.db.session import SessionMaker, make_engine, make_sessionmaker
from app.repo.git import UploadsRepo
from app.texts import Texts, load_texts
ROOT = Path(__file__).resolve().parent.parent
@@ -22,3 +29,65 @@ def settings(tmp_path: Path) -> Settings:
access=AccessSettings(admin_ids=frozenset({ADMIN})),
runtime=RuntimeSettings(config_dir=ROOT / "config", heartbeat_file=tmp_path / "hb"),
)
@pytest.fixture
async def engine() -> AsyncIterator[AsyncEngine]:
"""Fresh in-memory SQLite with all migrations applied — the same path the bot takes at startup."""
eng = make_engine("sqlite+aiosqlite://")
await upgrade(eng)
yield eng
await eng.dispose()
@pytest.fixture
def sessions(engine: AsyncEngine) -> SessionMaker:
return make_sessionmaker(engine)
@pytest.fixture
async def access(sessions: SessionMaker, settings: Settings) -> AccessService:
service = AccessService(sessions, settings.access)
await service.bootstrap_admins()
return service
def git(cwd: Path, *args: str) -> str:
env = {
"GIT_AUTHOR_NAME": "Тест",
"GIT_AUTHOR_EMAIL": "t@t",
"GIT_COMMITTER_NAME": "Тест",
"GIT_COMMITTER_EMAIL": "t@t",
"PATH": "/usr/bin:/bin",
"HOME": str(cwd),
}
return subprocess.run(["git", *args], cwd=cwd, env=env, check=True, capture_output=True, text=True).stdout
@pytest.fixture
def remote(tmp_path: Path) -> Path:
"""A bare 'SourceCraft' repo: services/hello, a file with a space and Cyrillic, a binary, 25 files in one dir."""
work = tmp_path / "seed"
work.mkdir()
git(work, "init", "-q", "-b", "main")
(work / "services" / "hello").mkdir(parents=True)
(work / "services" / "hello" / "main.py").write_text("print('hello')\n", encoding="utf-8")
(work / "README.md").write_text("# uploads\n", encoding="utf-8")
git(work, "add", "-A")
git(work, "commit", "-q", "-m", "init")
(work / "docs").mkdir()
(work / "docs" / "мои заметки.md").write_text("привет\n", encoding="utf-8")
(work / "docs" / "logo.bin").write_bytes(b"\x00\x01\x02binary")
(work / "many").mkdir()
for i in range(25):
(work / "many" / f"f{i:02}.txt").write_text(str(i), encoding="utf-8")
git(work, "add", "-A")
git(work, "commit", "-q", "-m", "docs and many")
bare = tmp_path / "remote.git"
git(tmp_path, "clone", "-q", "--bare", str(work), str(bare))
return bare
@pytest.fixture
def repo(remote: Path, tmp_path: Path) -> UploadsRepo:
return UploadsRepo(GitSettings(uploads_url=str(remote), clone_dir=tmp_path / "clone", fetch_interval=0))
View File
+30
View File
@@ -0,0 +1,30 @@
from datetime import timedelta
from sqlalchemy import func, select
from app.db.maintenance import cleanup_expired
from app.db.models import AccessAttempt, ContainerSnapshot, Draft, User, utcnow
from app.db.session import SessionMaker
async def test_cleanup_removes_only_expired(sessions: SessionMaker) -> None:
now = utcnow()
async with sessions.begin() as s:
s.add_all([User(tg_id=1, full_name="a"), User(tg_id=2, full_name="b")])
await s.flush() # no relationship() to order inserts: users first for the FK
s.add_all(
[
Draft(user_id=1, base_sha="a" * 40, expires_at=now - timedelta(minutes=1)),
Draft(user_id=2, base_sha="b" * 40, expires_at=now + timedelta(hours=1)),
ContainerSnapshot(container="c", at=now - timedelta(days=2), restart_count=0, cpu_pct=0, mem_pct=0),
ContainerSnapshot(container="c", at=now, restart_count=0, cpu_pct=0, mem_pct=0),
AccessAttempt(tg_id=9, at=now - timedelta(days=2), ok=False),
]
)
removed = await cleanup_expired(sessions, now)
assert removed == {"drafts": 1, "snapshots": 1, "access_attempts": 1}
async with sessions() as s:
assert await s.scalar(select(func.count()).select_from(Draft)) == 1
assert await s.scalar(select(func.count()).select_from(ContainerSnapshot)) == 1
+82
View File
@@ -0,0 +1,82 @@
from datetime import timedelta
import pytest
from alembic.autogenerate import compare_metadata
from alembic.migration import MigrationContext
from sqlalchemy import inspect, text
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncEngine
from app.db.migrate import downgrade, upgrade
from app.db.models import Base, ChangeOp, Draft, DraftChange, User, utcnow
from app.db.session import SessionMaker, make_engine
TABLES = {
"users",
"invites",
"access_attempts",
"fsm_state",
"drafts",
"draft_changes",
"pushes",
"pipeline_runs",
"container_snapshots",
"alerts",
"alert_deliveries",
"log_queries",
"agent_requests",
}
async def table_names(engine: AsyncEngine) -> set[str]:
async with engine.connect() as conn:
return set(await conn.run_sync(lambda c: inspect(c).get_table_names())) - {"alembic_version"}
async def test_upgrade_creates_all_tables(engine: AsyncEngine) -> None:
assert await table_names(engine) == TABLES
async def test_models_match_migrations(engine: AsyncEngine) -> None:
# A model change without a migration shows up here as a non-empty diff.
async with engine.connect() as conn:
diff = await conn.run_sync(lambda c: compare_metadata(MigrationContext.configure(c), Base.metadata))
assert diff == []
async def test_downgrade_to_base_and_back() -> None:
eng = make_engine("sqlite+aiosqlite://")
try:
await upgrade(eng)
await downgrade(eng, "base")
assert await table_names(eng) == set()
await upgrade(eng)
assert await table_names(eng) == TABLES
finally:
await eng.dispose()
INSERT_USER = text(
"INSERT INTO users (tg_id, full_name, role, is_active, joined_at) VALUES (:id, 'u', :role, 1, '2026-01-01')"
)
async def test_enum_check_rejects_garbage(engine: AsyncEngine) -> None:
async with engine.begin() as conn:
await conn.execute(INSERT_USER, {"id": 1, "role": "admin"})
with pytest.raises(IntegrityError):
async with engine.begin() as conn:
await conn.execute(INSERT_USER, {"id": 2, "role": "root"})
async def test_deleting_draft_cascades_to_changes(sessions: SessionMaker) -> None:
async with sessions.begin() as s:
s.add(User(tg_id=1, full_name="a"))
await s.flush()
draft = Draft(user_id=1, base_sha="0" * 40, expires_at=utcnow() + timedelta(hours=1))
draft.changes.append(DraftChange(path="services/a/main.py", op=ChangeOp.ADD, content=b"print(1)"))
s.add(draft)
async with sessions.begin() as s:
await s.execute(text("DELETE FROM drafts"))
async with sessions() as s:
assert (await s.execute(text("SELECT count(*) FROM draft_changes"))).scalar_one() == 0
+32 -5
View File
@@ -7,9 +7,9 @@ from typing import Any
from aiogram import Bot, Dispatcher
from aiogram.client.session.base import BaseSession
from aiogram.methods import EditMessageText, SendMessage, TelegramMethod
from aiogram.methods import EditMessageText, GetFile, GetMe, SendMessage, SendPhoto, TelegramMethod
from aiogram.methods.base import TelegramType
from aiogram.types import Chat, Message, Update, User
from aiogram.types import Chat, File, Message, Update, User
BOT_TOKEN = "42:TEST-TOKEN"
@@ -19,6 +19,7 @@ class FakeSession(BaseSession):
super().__init__()
self.requests: list[TelegramMethod[Any]] = []
self._ids = count(1000)
self.files: dict[str, bytes] = {} # file_id -> content, for documents users "upload"
async def make_request(
self,
@@ -27,6 +28,20 @@ class FakeSession(BaseSession):
timeout: int | None = None, # noqa: ASYNC109 - signature fixed by aiogram BaseSession
) -> TelegramType:
self.requests.append(method)
if isinstance(method, GetFile):
return File( # type: ignore[return-value]
file_id=method.file_id, file_unique_id=method.file_id, file_path=f"documents/{method.file_id}"
)
if isinstance(method, GetMe):
return User(id=42, is_bot=True, first_name="tgit", username="tgit_test_bot") # type: ignore[return-value]
if isinstance(method, SendPhoto):
chat = method.chat_id if isinstance(method.chat_id, int) else 0
return Message( # type: ignore[return-value]
message_id=next(self._ids),
date=datetime.now(UTC),
chat=Chat(id=chat, type="private"),
caption=method.caption,
)
if isinstance(method, SendMessage | EditMessageText):
chat_id = method.chat_id if isinstance(method.chat_id, int) else 0
return Message( # type: ignore[return-value]
@@ -44,9 +59,9 @@ class FakeSession(BaseSession):
timeout: int = 30, # noqa: ASYNC109 - signature fixed by aiogram BaseSession
chunk_size: int = 65536,
raise_for_status: bool = True,
) -> AsyncGenerator[bytes, None]: # pragma: no cover - not used
raise NotImplementedError
yield b""
) -> AsyncGenerator[bytes, None]:
file_id = url.rsplit("/", 1)[-1]
yield self.files[file_id]
async def close(self) -> None:
pass
@@ -82,6 +97,18 @@ class TgClient:
message["entities"] = [{"type": "bot_command", "offset": 0, "length": len(cmd)}]
await self._feed({"message": message})
async def send_document(self, tg_id: int, name: str, content: bytes) -> None:
file_id = f"doc{next(self._message_ids)}"
self.session.files[file_id] = content
message = {
"message_id": next(self._message_ids),
"date": int(datetime.now(UTC).timestamp()),
"chat": {"id": tg_id, "type": "private"},
"from": self.user(tg_id),
"document": {"file_id": file_id, "file_unique_id": file_id, "file_name": name, "file_size": len(content)},
}
await self._feed({"message": message})
async def press(self, tg_id: int, data: str, message_text: str = "section") -> None:
callback = {
"id": str(next(self._update_ids)),
+5
View File
@@ -0,0 +1,5 @@
{
"logs": "Downloading ruff (12.8MiB)\n Downloading ruff\nInstalled 1 package in 1ms\nF401 [*] `os` imported but unused\n --> services/broken/main.py:1:8\n |\n1 | import os\n | ^^\n2 |\n3 | def run():\n |\nhelp: Remove unused import: `os`\n\nF821 Undefined name `undefined_name`\n --> services/broken/main.py:4:12\n |\n3 | def run():\n4 | return undefined_name\n | ^^^^^^^^^^^^^^\n |\n\nFound 2 errors.\n[*] 1 fixable with the `--fix` option.\n",
"page_complete": false,
"done": true
}
+5
View File
@@ -0,0 +1,5 @@
{
"logs": "TGIT_COMMIT_SHA=932f4b1a144cfed4fed8eafd94433e0995e5b12d\n",
"page_complete": false,
"done": true
}
+396
View File
@@ -0,0 +1,396 @@
{
"id": "",
"slug": "1000001",
"dates": {
"created_at": "2026-10-13T20:58:52.273658Z",
"started_at": "2026-10-13T20:58:53.940173Z",
"finished_at": "2026-10-13T20:59:17.833911Z",
"updated_at": "2026-10-13T20:59:17.867686Z"
},
"status": "failed",
"workflows": [
{
"id": "",
"slug": "check-and-deploy",
"description": "",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "2026-10-13T20:58:53.940173Z",
"finished_at": "2026-10-13T20:59:17.833911Z",
"updated_at": "2026-10-13T20:59:17.862572Z"
},
"status": "failed",
"tasks": [
{
"id": "",
"slug": "check",
"description": "",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "2026-10-13T20:58:53.940173Z",
"finished_at": "2026-10-13T20:59:11.150164Z",
"updated_at": "2026-10-13T20:59:12.846332Z"
},
"status": "failed",
"cubes": [
{
"id": "",
"slug": "#create-environment",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "2026-10-13T20:58:54.031424Z",
"finished_at": "2026-10-13T20:58:54.090132Z",
"updated_at": "2026-10-13T20:58:54.155008Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": []
}
},
{
"id": "",
"slug": "#git-clone",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "2026-10-13T20:58:54.090481Z",
"finished_at": "2026-10-13T20:58:55.142644Z",
"updated_at": "2026-10-13T20:58:55.211046Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#create-environment"
}
]
}
},
{
"id": "",
"slug": "#prepare-image-for-compile",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "2026-10-13T20:58:55.143033Z",
"finished_at": "2026-10-13T20:58:59.605194Z",
"updated_at": "2026-10-13T20:58:59.706155Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#git-clone"
}
]
}
},
{
"id": "",
"slug": "#prepare-image-for-lint",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "2026-10-13T20:58:59.605538Z",
"finished_at": "2026-10-13T20:59:00.536788Z",
"updated_at": "2026-10-13T20:59:00.604498Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#git-clone"
}
]
}
},
{
"id": "",
"slug": "#prepare-image-for-security",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "2026-10-13T20:59:00.537171Z",
"finished_at": "2026-10-13T20:59:01.447150Z",
"updated_at": "2026-10-13T20:59:01.511994Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#git-clone"
}
]
}
},
{
"id": "",
"slug": "#prepare-image-for-tests",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "2026-10-13T20:59:01.447481Z",
"finished_at": "2026-10-13T20:59:02.624921Z",
"updated_at": "2026-10-13T20:59:02.700313Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#git-clone"
}
]
}
},
{
"id": "",
"slug": "compile",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "2026-10-13T20:59:02.625267Z",
"finished_at": "2026-10-13T20:59:09.366621Z",
"updated_at": "2026-10-13T20:59:09.454706Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-image-for-compile"
}
]
}
},
{
"id": "",
"slug": "lint",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "2026-10-13T20:59:09.366953Z",
"finished_at": "2026-10-13T20:59:11.088201Z",
"updated_at": "2026-10-13T20:59:11.173518Z"
},
"status": "failed",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-image-for-lint"
}
]
}
},
{
"id": "",
"slug": "security",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "2026-10-13T20:59:11.088571Z",
"updated_at": "2026-10-13T20:59:11.206851Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-image-for-security"
}
]
}
},
{
"id": "",
"slug": "tests",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "2026-10-13T20:59:11.088804Z",
"updated_at": "2026-10-13T20:59:11.242177Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-image-for-tests"
}
]
}
},
{
"id": "",
"slug": "#tear-down-environment",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "2026-10-13T20:59:11.089254Z",
"finished_at": "2026-10-13T20:59:11.150164Z",
"updated_at": "2026-10-13T20:59:11.346169Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "compile"
},
{
"name": "lint"
},
{
"name": "security"
},
{
"name": "tests"
}
]
}
}
],
"relations": {
"needs": []
}
},
{
"id": "",
"slug": "deploy",
"description": "",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "2026-10-13T20:59:17.833911Z",
"updated_at": "2026-10-13T20:59:17.840273Z"
},
"status": "created",
"cubes": [
{
"id": "",
"slug": "#create-environment",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-13T20:58:52.727666Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": []
}
},
{
"id": "",
"slug": "#git-clone",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-13T20:58:52.727666Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#create-environment"
}
]
}
},
{
"id": "",
"slug": "#prepare-artifacts",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-13T20:58:52.727666Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#git-clone"
}
]
}
},
{
"id": "",
"slug": "#prepare-image-for-deploy",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-13T20:58:52.727666Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-artifacts"
}
]
}
},
{
"id": "",
"slug": "deploy",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-13T20:58:52.727666Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-image-for-deploy"
}
]
}
},
{
"id": "",
"slug": "#tear-down-environment",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-13T20:58:52.727666Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": [
{
"name": "deploy"
}
]
}
}
],
"relations": {
"needs": [
{
"name": "check"
}
]
}
}
],
"environment_id": ""
}
],
"event_type": "manual",
"error_messages": [],
"user": {
"id": "01a06158-1562-7db1-b6e9-018ddd4d4c29",
"slug": "rogachevegorushka"
}
}
+437
View File
@@ -0,0 +1,437 @@
{
"id": "",
"slug": "1000002",
"dates": {
"created_at": "2026-10-14T14:51:56.652199Z",
"started_at": "2026-10-14T14:57:43.223468Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-14T14:57:43.242955Z"
},
"status": "processing",
"workflows": [
{
"id": "",
"slug": "check-and-deploy",
"description": "",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:57:43.223468Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-14T14:57:43.234813Z"
},
"status": "processing",
"tasks": [
{
"id": "",
"slug": "check",
"description": "",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:57:43.223468Z",
"finished_at": "2026-10-14T14:58:19.375134Z",
"updated_at": "2026-10-14T14:58:19.571691Z"
},
"status": "success",
"cubes": [
{
"id": "",
"slug": "#create-environment",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:57:43.514526Z",
"finished_at": "2026-10-14T14:57:43.584723Z",
"updated_at": "2026-10-14T14:57:43.657871Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": []
}
},
{
"id": "",
"slug": "#git-clone",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:57:43.585127Z",
"finished_at": "2026-10-14T14:57:45.528681Z",
"updated_at": "2026-10-14T14:57:45.606302Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#create-environment"
}
]
}
},
{
"id": "",
"slug": "#prepare-image-for-compile",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:57:45.529064Z",
"finished_at": "2026-10-14T14:57:55.435126Z",
"updated_at": "2026-10-14T14:57:55.537030Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#git-clone"
}
]
}
},
{
"id": "",
"slug": "#prepare-image-for-lint",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:57:55.435513Z",
"finished_at": "2026-10-14T14:57:58.411834Z",
"updated_at": "2026-10-14T14:57:58.479777Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#git-clone"
}
]
}
},
{
"id": "",
"slug": "#prepare-image-for-meta",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:57:58.412252Z",
"finished_at": "2026-10-14T14:58:03.323469Z",
"updated_at": "2026-10-14T14:58:03.420479Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#git-clone"
}
]
}
},
{
"id": "",
"slug": "#prepare-image-for-security",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:58:03.323798Z",
"finished_at": "2026-10-14T14:58:04.898405Z",
"updated_at": "2026-10-14T14:58:04.986588Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#git-clone"
}
]
}
},
{
"id": "",
"slug": "#prepare-image-for-tests",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:58:04.898847Z",
"finished_at": "2026-10-14T14:58:09.732911Z",
"updated_at": "2026-10-14T14:58:09.807301Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#git-clone"
}
]
}
},
{
"id": "",
"slug": "compile",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:58:09.733227Z",
"finished_at": "2026-10-14T14:58:12.400619Z",
"updated_at": "2026-10-14T14:58:12.533056Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-image-for-compile"
}
]
}
},
{
"id": "",
"slug": "lint",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:58:12.401008Z",
"finished_at": "2026-10-14T14:58:14.235114Z",
"updated_at": "2026-10-14T14:58:14.304352Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-image-for-lint"
}
]
}
},
{
"id": "",
"slug": "meta",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:58:14.235485Z",
"finished_at": "2026-10-14T14:58:14.707956Z",
"updated_at": "2026-10-14T14:58:14.778120Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-image-for-meta"
}
]
}
},
{
"id": "",
"slug": "security",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:58:14.708362Z",
"finished_at": "2026-10-14T14:58:17.148819Z",
"updated_at": "2026-10-14T14:58:17.221280Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-image-for-security"
}
]
}
},
{
"id": "",
"slug": "tests",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:58:17.149213Z",
"finished_at": "2026-10-14T14:58:19.298832Z",
"updated_at": "2026-10-14T14:58:19.405984Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-image-for-tests"
}
]
}
},
{
"id": "",
"slug": "#tear-down-environment",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:58:19.299366Z",
"finished_at": "2026-10-14T14:58:19.375117Z",
"updated_at": "2026-10-14T14:58:19.528162Z"
},
"status": "success",
"artifacts": [],
"relations": {
"needs": [
{
"name": "compile"
},
{
"name": "lint"
},
{
"name": "meta"
},
{
"name": "security"
},
{
"name": "tests"
}
]
}
}
],
"relations": {
"needs": []
}
},
{
"id": "",
"slug": "deploy",
"description": "",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-14T14:58:21.737007Z"
},
"status": "created",
"cubes": [
{
"id": "",
"slug": "#create-environment",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-14T14:51:57.522605Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": []
}
},
{
"id": "",
"slug": "#git-clone",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-14T14:51:57.522605Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#create-environment"
}
]
}
},
{
"id": "",
"slug": "#prepare-artifacts",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-14T14:51:57.522605Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#git-clone"
}
]
}
},
{
"id": "",
"slug": "#prepare-image-for-deploy",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-14T14:51:57.522605Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-artifacts"
}
]
}
},
{
"id": "",
"slug": "deploy",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-14T14:51:57.522605Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": [
{
"name": "#prepare-image-for-deploy"
}
]
}
},
{
"id": "",
"slug": "#tear-down-environment",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "1970-01-01T00:00:00Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-14T14:51:57.522605Z"
},
"status": "created",
"artifacts": [],
"relations": {
"needs": [
{
"name": "deploy"
}
]
}
}
],
"relations": {
"needs": [
{
"name": "check"
}
]
}
}
],
"environment_id": ""
}
],
"event_type": "push",
"error_messages": [],
"user": {
"id": "01a06158-1562-7db1-b6e9-018ddd4d4c29",
"slug": "rogachevegorushka"
}
}
+104
View File
@@ -0,0 +1,104 @@
{
"runs": [
{
"id": "",
"slug": "1000002",
"dates": {
"created_at": "2026-10-14T14:51:56.652199Z",
"started_at": "2026-10-14T14:57:43.223468Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-14T14:57:43.242955Z"
},
"status": "processing",
"workflows": [
{
"id": "",
"slug": "check-and-deploy",
"description": "",
"dates": {
"created_at": "2026-10-14T14:51:57.521264Z",
"started_at": "2026-10-14T14:57:43.223468Z",
"finished_at": "1970-01-01T00:00:00Z",
"updated_at": "2026-10-14T14:57:43.234813Z"
},
"status": "processing",
"tasks": [],
"environment_id": ""
}
],
"event_type": "push",
"error_messages": [],
"user": {
"id": "01a06158-1562-7db1-b6e9-018ddd4d4c29",
"slug": "rogachevegorushka"
}
},
{
"id": "",
"slug": "1000001",
"dates": {
"created_at": "2026-10-13T20:58:52.273658Z",
"started_at": "2026-10-13T20:58:53.940173Z",
"finished_at": "2026-10-13T20:59:17.833911Z",
"updated_at": "2026-10-13T20:59:17.867686Z"
},
"status": "failed",
"workflows": [
{
"id": "",
"slug": "check-and-deploy",
"description": "",
"dates": {
"created_at": "2026-10-13T20:58:52.726601Z",
"started_at": "2026-10-13T20:58:53.940173Z",
"finished_at": "2026-10-13T20:59:17.833911Z",
"updated_at": "2026-10-13T20:59:17.862572Z"
},
"status": "failed",
"tasks": [],
"environment_id": ""
}
],
"event_type": "manual",
"error_messages": [],
"user": {
"id": "01a06158-1562-7db1-b6e9-018ddd4d4c29",
"slug": "rogachevegorushka"
}
},
{
"id": "",
"slug": "1000000",
"dates": {
"created_at": "2026-10-13T20:57:06.596315Z",
"started_at": "2026-10-13T20:57:06.821003Z",
"finished_at": "2026-10-13T20:58:26.698635Z",
"updated_at": "2026-10-13T20:58:27.884131Z"
},
"status": "success",
"workflows": [
{
"id": "",
"slug": "check-and-deploy",
"description": "",
"dates": {
"created_at": "2026-10-13T20:57:06.725971Z",
"started_at": "2026-10-13T20:57:06.821003Z",
"finished_at": "2026-10-13T20:58:26.698635Z",
"updated_at": "2026-10-13T20:58:27.878032Z"
},
"status": "success",
"tasks": [],
"environment_id": ""
}
],
"event_type": "push",
"error_messages": [],
"user": {
"id": "01a06158-1562-7db1-b6e9-018ddd4d4c29",
"slug": "rogachevegorushka"
}
}
],
"next_page_token": "147400973399132381"
}
View File
+72
View File
@@ -0,0 +1,72 @@
from datetime import UTC, datetime, timedelta
import pytest
from app.db.models import ChangeOp, User
from app.db.session import SessionMaker
from app.repo.drafts import MAX_CHANGES, DraftFull, DraftService
UID = 7
SHA = "a" * 40
class Clock:
now = datetime(2026, 10, 15, tzinfo=UTC)
def __call__(self) -> datetime:
return self.now
@pytest.fixture
async def svc(sessions: SessionMaker) -> DraftService:
async with sessions.begin() as s:
s.add(User(tg_id=UID, full_name="u"))
return DraftService(sessions, clock=Clock())
async def test_put_get_and_order(svc: DraftService) -> None:
await svc.put(UID, SHA, "b.py", ChangeOp.ADD, b"b")
await svc.put(UID, "other-sha", "a.py", ChangeOp.MODIFY, b"a")
draft = await svc.get(UID)
assert draft is not None and draft.base_sha == SHA # base fixed when the draft started
assert [(c.path, c.op) for c in draft.changes] == [("b.py", ChangeOp.ADD), ("a.py", ChangeOp.MODIFY)]
async def test_editing_a_new_file_keeps_it_added_and_deleting_it_drops_it(svc: DraftService) -> None:
await svc.put(UID, SHA, "n.py", ChangeOp.ADD, b"1")
await svc.put(UID, SHA, "n.py", ChangeOp.MODIFY, b"2")
draft = await svc.get(UID)
assert draft is not None and draft.changes[0].op is ChangeOp.ADD and draft.changes[0].content == b"2"
await svc.put(UID, SHA, "n.py", ChangeOp.DELETE, None)
draft = await svc.get(UID)
assert draft is not None and draft.changes == []
async def test_drop_message_clear(svc: DraftService) -> None:
await svc.put(UID, SHA, "a.py", ChangeOp.ADD, b"1")
await svc.put(UID, SHA, "b.py", ChangeOp.DELETE, None)
await svc.drop(UID, "a.py")
await svc.set_message(UID, "feat: x")
draft = await svc.get(UID)
assert draft is not None and [c.path for c in draft.changes] == ["b.py"] and draft.message == "feat: x"
await svc.clear(UID)
assert await svc.get(UID) is None
async def test_expired_draft_is_gone(svc: DraftService) -> None:
await svc.put(UID, SHA, "a.py", ChangeOp.ADD, b"1")
Clock.now += timedelta(hours=25)
try:
assert await svc.get(UID) is None
finally:
Clock.now -= timedelta(hours=25)
async def test_draft_size_limit(svc: DraftService) -> None:
for i in range(MAX_CHANGES):
await svc.put(UID, SHA, f"f{i}.py", ChangeOp.ADD, b"")
with pytest.raises(DraftFull):
await svc.put(UID, SHA, "one-more.py", ChangeOp.ADD, b"")
+96
View File
@@ -0,0 +1,96 @@
from pathlib import Path
import pytest
from app.repo.git import GitError, UploadsRepo
from tests.conftest import git
async def test_clone_and_list_root_dirs_first(repo: UploadsRepo) -> None:
await repo.refresh()
entries = await repo.list_dir()
assert [(e.name, e.kind) for e in entries] == [
("docs", "dir"),
("many", "dir"),
("services", "dir"),
("README.md", "file"),
]
assert entries[-1].size == len("# uploads\n")
async def test_unicode_and_spaces_in_names(repo: UploadsRepo) -> None:
await repo.refresh()
names = [e.path for e in await repo.list_dir("docs")]
assert names == ["docs/logo.bin", "docs/мои заметки.md"]
assert (await repo.read("docs/мои заметки.md")).decode() == "привет\n"
async def test_last_commit_per_path(repo: UploadsRepo) -> None:
await repo.refresh()
hello = await repo.last_commit("services/hello/main.py")
docs = await repo.last_commit("docs/logo.bin")
assert hello is not None and hello.subject == "init" and hello.author == "Тест"
assert docs is not None and docs.subject == "docs and many" and len(docs.sha) == 40
async def test_missing_dir_is_an_error(repo: UploadsRepo) -> None:
await repo.refresh()
with pytest.raises(GitError, match="no such directory"):
await repo.list_dir("nope")
async def test_refresh_sees_new_commits_and_respects_interval(repo: UploadsRepo, remote: Path, tmp_path: Path) -> None:
await repo.refresh()
before = await repo.head()
other = tmp_path / "other"
git(tmp_path, "clone", "-q", str(remote), str(other))
(other / "new.txt").write_text("x", encoding="utf-8")
git(other, "add", "-A")
git(other, "commit", "-q", "-m", "new")
git(other, "push", "-q", "origin", "main")
await repo.refresh()
assert await repo.head() != before
assert "new.txt" in [e.name for e in await repo.list_dir()]
async def test_fetch_is_throttled(remote: Path, tmp_path: Path) -> None:
from app.config import GitSettings
throttled = UploadsRepo(GitSettings(uploads_url=str(remote), clone_dir=tmp_path / "c2", fetch_interval=3600))
await throttled.refresh()
head = await throttled.head()
other = tmp_path / "o2"
git(tmp_path, "clone", "-q", str(remote), str(other))
(other / "x.txt").write_text("x", encoding="utf-8")
git(other, "add", "-A")
git(other, "commit", "-q", "-m", "x")
git(other, "push", "-q", "origin", "main")
await throttled.refresh()
assert await throttled.head() == head # within the interval: no fetch
await throttled.refresh(force=True)
assert await throttled.head() != head
async def test_unreachable_remote(tmp_path: Path) -> None:
from app.config import GitSettings
broken = UploadsRepo(GitSettings(uploads_url=str(tmp_path / "missing.git"), clone_dir=tmp_path / "c3"))
with pytest.raises(GitError, match="clone"):
await broken.refresh()
async def test_relative_clone_dir(remote: Path, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
from app.config import GitSettings
monkeypatch.chdir(tmp_path)
relative = UploadsRepo(GitSettings(uploads_url=str(remote), clone_dir=Path("data/uploads")))
await relative.refresh()
assert (tmp_path / "data" / "uploads" / ".git").is_dir()
assert "README.md" in [e.name for e in await relative.list_dir()]
+143
View File
@@ -0,0 +1,143 @@
from datetime import timedelta
from pathlib import Path
import pytest
from sqlalchemy import select
from app.access.service import AccessService
from app.db.models import ChangeOp, Push, User, utcnow
from app.db.session import SessionMaker
from app.repo.drafts import Change, DraftView
from app.repo.git import UploadsRepo
from app.repo.publisher import PUSHES_PER_HOUR, Publisher, PushStatus
from tests.conftest import ADMIN, git
@pytest.fixture
async def member(access: AccessService) -> User:
user = await access.get_member(ADMIN)
assert user is not None
return user
def remote_log(remote: Path, tmp_path: Path, name: str = "check") -> tuple[Path, str]:
clone = tmp_path / name
git(tmp_path, "clone", "-q", str(remote), str(clone))
return clone, git(clone, "log", "-1", "--format=%an <%ae>%n%B")
def foreign_commit(remote: Path, tmp_path: Path, path: str, text: str, name: str) -> None:
other = tmp_path / name
git(tmp_path, "clone", "-q", str(remote), str(other))
target = other / path
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(text, encoding="utf-8")
git(other, "add", "-A")
git(other, "commit", "-q", "-m", f"foreign {path}")
git(other, "push", "-q", "origin", "main")
async def draft_on(repo: UploadsRepo, *changes: Change) -> DraftView:
await repo.refresh(force=True)
return DraftView(base_sha=await repo.head(), message=None, changes=list(changes))
async def test_push_applies_changes_with_trailer(
repo: UploadsRepo, sessions: SessionMaker, member: User, remote: Path, tmp_path: Path
) -> None:
draft = await draft_on(
repo,
Change("services/hello/utils.py", ChangeOp.ADD, b"X = 1\n"),
Change("services/hello/main.py", ChangeOp.MODIFY, b"print('v2')\n"),
Change("docs/logo.bin", ChangeOp.DELETE, None),
Change("docs/мои заметки.md", ChangeOp.DELETE, None),
)
publisher = Publisher(repo, sessions)
result = await publisher.push(member, draft, "feat: v2\n\nподробности")
assert result.status is PushStatus.OK and result.sha is not None
clone, log = remote_log(remote, tmp_path)
assert log.startswith("tgit-bot <tgit-bot@users.noreply.tgit>\nfeat: v2\n\nподробности\n\nTelegram-User: ")
assert f"({ADMIN})" in log
assert (clone / "services/hello/utils.py").read_text() == "X = 1\n"
assert (clone / "services/hello/main.py").read_text() == "print('v2')\n"
assert not (clone / "docs").exists() # both files gone → the folder is gone too
assert git(clone, "rev-parse", "HEAD").strip() == result.sha
async def test_nothing_to_commit(repo: UploadsRepo, sessions: SessionMaker, member: User) -> None:
draft = await draft_on(repo, Change("README.md", ChangeOp.MODIFY, b"# uploads\n"))
assert (await Publisher(repo, sessions).push(member, draft, "noop")).status is PushStatus.EMPTY
async def test_conflict_then_overwrite(
repo: UploadsRepo, sessions: SessionMaker, member: User, remote: Path, tmp_path: Path
) -> None:
draft = await draft_on(repo, Change("README.md", ChangeOp.MODIFY, b"mine\n"))
foreign_commit(remote, tmp_path, "README.md", "theirs\n", "other")
publisher = Publisher(repo, sessions)
conflict = await publisher.push(member, draft, "edit readme")
assert conflict.status is PushStatus.CONFLICT and conflict.conflicts == ["README.md"]
forced = await publisher.push(member, draft, "edit readme", overwrite=True)
assert forced.status is PushStatus.OK
clone, _ = remote_log(remote, tmp_path)
assert (clone / "README.md").read_text() == "mine\n"
async def test_unrelated_change_in_main_is_not_a_conflict(
repo: UploadsRepo, sessions: SessionMaker, member: User, remote: Path, tmp_path: Path
) -> None:
draft = await draft_on(repo, Change("a.txt", ChangeOp.ADD, b"a"))
foreign_commit(remote, tmp_path, "b.txt", "b", "other")
assert (await Publisher(repo, sessions).push(member, draft, "add a")).status is PushStatus.OK
clone, _ = remote_log(remote, tmp_path)
assert (clone / "a.txt").exists() and (clone / "b.txt").exists()
async def test_rejected_push_is_rebased(
repo: UploadsRepo, sessions: SessionMaker, member: User, remote: Path, tmp_path: Path
) -> None:
raced = False
async def someone_pushes_first() -> None:
nonlocal raced
if not raced:
raced = True
foreign_commit(remote, tmp_path, "race.txt", "r", "racer")
draft = await draft_on(repo, Change("mine.txt", ChangeOp.ADD, b"m"))
result = await Publisher(repo, sessions, before_push=someone_pushes_first).push(member, draft, "add mine")
assert result.status is PushStatus.OK and raced
clone, log = remote_log(remote, tmp_path)
assert "add mine" in log and (clone / "race.txt").exists() and (clone / "mine.txt").exists()
async def test_path_escape_is_refused(repo: UploadsRepo, sessions: SessionMaker, member: User, tmp_path: Path) -> None:
draft = await draft_on(repo, Change("../evil.txt", ChangeOp.ADD, b"x"))
assert (await Publisher(repo, sessions).push(member, draft, "evil")).status is PushStatus.FAILED
assert not (tmp_path / "evil.txt").exists()
async def test_rate_limit_and_record(repo: UploadsRepo, sessions: SessionMaker, member: User) -> None:
publisher = Publisher(repo, sessions)
draft = await draft_on(repo, Change("x.txt", ChangeOp.ADD, b"x"))
result = await publisher.push(member, draft, "x")
await publisher.record(member, result, "x", draft, chat_id=ADMIN, message_id=77)
async with sessions() as s:
push = await s.scalar(select(Push))
assert push is not None and push.sha == result.sha and push.paths == ["x.txt"] and push.message_id == 77
for i in range(PUSHES_PER_HOUR):
s.add(
Push(user_id=ADMIN, sha=f"{i:040d}", message="m", chat_id=1, created_at=utcnow() - timedelta(minutes=1))
)
await s.commit()
assert (await publisher.push(member, draft, "y")).status is PushStatus.LIMIT
+74
View File
@@ -0,0 +1,74 @@
import pytest
from app.repo.validate import (
ValidationError,
check_content,
check_name,
diff_stat,
is_protected,
join,
python_syntax,
strip_fence,
)
@pytest.mark.parametrize("name", ["main.py", "a-b_c.1", ".gitignore", "X" * 100])
def test_good_names(name: str) -> None:
assert check_name(f" {name} ") == name
@pytest.mark.parametrize("name", ["", ".", "..", "a/b", "привет.py", "a b", "X" * 101, "a;rm"])
def test_bad_names(name: str) -> None:
with pytest.raises(ValidationError):
check_name(name)
@pytest.mark.parametrize(
("path", "protected"),
[
(".sourcecraft/ci.yaml", True),
(".tgit/x.py", True),
("ruff.toml", True),
(".sourcecraft", True),
("services/a/ruff.toml", False),
("README.md", False),
("services/a/main.py", False),
],
)
def test_protected_paths(path: str, protected: bool) -> None:
assert is_protected(path) is protected
def test_join() -> None:
assert join("", "a.py") == "a.py" and join("s/x", "a.py") == "s/x/a.py"
@pytest.mark.parametrize(
("text", "expected"),
[
("```python\nprint(1)\n```", "print(1)"),
("```\na\nb\n```", "a\nb"),
("print(1)", "print(1)"),
("x ``` y", "x ``` y"),
],
)
def test_strip_fence(text: str, expected: str) -> None:
assert strip_fence(text) == expected
def test_content_checks() -> None:
assert check_content("привет".encode()) == "привет"
for data, reason in [(b"a\0b", "binary"), (b"\xff\xfe", "encoding"), (b"x" * (100 * 1024 + 1), "too_big")]:
with pytest.raises(ValidationError, match=reason):
check_content(data)
def test_python_syntax_only_for_py() -> None:
issue = python_syntax("a.py", "def f(:\n pass\n")
assert issue is not None and issue.line == 1
assert python_syntax("a.py", "x = 1\n") is None
assert python_syntax("a.txt", "def f(:") is None
def test_diff_stat() -> None:
assert diff_stat("a\nb\nc", "a\nB\nc\nd") == (2, 1)
View File
+160
View File
@@ -0,0 +1,160 @@
"""An in-memory Docker Engine API behind httpx.MockTransport, GET-only like docker-socket-proxy."""
import hashlib
import json
import struct
from dataclasses import dataclass, field
from datetime import UTC, datetime, timedelta
from typing import Any
from urllib.parse import parse_qs
import httpx
from app.config import DockerSettings
from app.runtime.docker import DockerClient
STARTED = datetime(2026, 10, 17, 8, 0, tzinfo=UTC)
MiB = 1024 * 1024
def frame(text: str, stream: int = 1) -> bytes:
data = text.encode()
return struct.pack(">BxxxI", stream, len(data)) + data
@dataclass
class FakeContainer:
id: str
name: str
service: str = ""
state: str = "running"
health: str | None = "healthy"
exit_code: int = 0
restart_count: int = 0
oom: bool = False
managed: bool = True
labels: dict[str, str] = field(default_factory=dict)
logs: list[tuple[datetime, str]] = field(default_factory=list) # (when, line)
cpu_pct: float = 5.0
mem_bytes: int = 64 * MiB
mem_limit: int = 256 * MiB
stats_status: int = 200
def all_labels(self) -> dict[str, str]:
base = {"com.docker.compose.service": self.service} if self.service else {}
return {**base, **({"tgit.managed": "true"} if self.managed else {}), **self.labels}
def status_line(self) -> str:
if self.state == "running":
return "Up 2 hours" + {
"healthy": " (healthy)",
"unhealthy": " (unhealthy)",
"starting": " (health: starting)",
}.get(self.health or "", "")
if self.state == "exited":
return f"Exited ({self.exit_code}) 5 minutes ago"
return self.state.capitalize()
class FakeDocker:
def __init__(self) -> None:
self.containers: dict[str, FakeContainer] = {}
self.requests: list[httpx.Request] = []
self.down = False
def add(self, name: str, **kw: Any) -> FakeContainer:
cid = hashlib.sha256(name.encode()).hexdigest()
c = FakeContainer(id=cid, name=name, **kw)
self.containers[cid] = c
return c
def client(self, **settings: Any) -> DockerClient:
return DockerClient(
DockerSettings(url="http://docker-proxy:2375", **settings), transport=httpx.MockTransport(self)
)
def __call__(self, request: httpx.Request) -> httpx.Response:
self.requests.append(request)
if self.down:
raise httpx.ConnectError("connection refused", request=request)
if request.method != "GET":
return httpx.Response(403, text="forbidden by proxy")
path = request.url.path
q = {k: v[0] for k, v in parse_qs(request.url.query.decode()).items()}
if path == "/_ping":
return httpx.Response(200, text="OK")
if path == "/containers/json":
return httpx.Response(200, json=self._list(q))
parts = path.strip("/").split("/")
if len(parts) == 3 and parts[0] == "containers" and parts[1] in self.containers:
c = self.containers[parts[1]]
match parts[2]:
case "json":
return httpx.Response(200, json=self._inspect(c))
case "stats":
if c.stats_status != 200:
return httpx.Response(c.stats_status, text="stats broken")
return httpx.Response(200, json=self._stats(c))
case "logs":
since = datetime.fromtimestamp(int(q.get("since", "0")), UTC)
body = b"".join(frame(line + "\n", 2 if "ERR" in line else 1) for at, line in c.logs if at >= since)
return httpx.Response(200, content=body)
return httpx.Response(404, json={"message": "No such container"})
def _list(self, q: dict[str, str]) -> list[dict[str, Any]]:
wanted = json.loads(q.get("filters", "{}")).get("label", [])
out = []
for c in self.containers.values():
labels = c.all_labels()
if all(labels.get(k) == v for k, _, v in (w.partition("=") for w in wanted)):
out.append(
{
"Id": c.id,
"Names": [f"/{c.name}"],
"Image": f"{c.name}:latest",
"State": c.state,
"Status": c.status_line(),
"Labels": labels,
}
)
return out
def _inspect(self, c: FakeContainer) -> dict[str, Any]:
state: dict[str, Any] = {
"Status": c.state,
"ExitCode": c.exit_code,
"OOMKilled": c.oom,
"StartedAt": STARTED.strftime("%Y-%m-%dT%H:%M:%S.123456789Z"),
"FinishedAt": "0001-01-01T00:00:00Z",
}
if c.health is not None:
state["Health"] = {"Status": c.health}
return {
"Id": c.id,
"Name": f"/{c.name}",
"RestartCount": c.restart_count,
"State": state,
"Config": {"Image": f"{c.name}:latest", "Labels": c.all_labels()},
}
@staticmethod
def _stats(c: FakeContainer) -> dict[str, Any]:
# 2 CPUs, system delta 1e9 → cpu_pct = delta/1e9*2*100
delta = int(c.cpu_pct / 200 * 1e9)
return {
"cpu_stats": {
"cpu_usage": {"total_usage": 10_000 + delta},
"system_cpu_usage": 2_000_000_000,
"online_cpus": 2,
},
"precpu_stats": {"cpu_usage": {"total_usage": 10_000}, "system_cpu_usage": 1_000_000_000},
"memory_stats": {
"usage": c.mem_bytes + 10 * MiB,
"limit": c.mem_limit,
"stats": {"inactive_file": 10 * MiB},
},
}
def now_minus(minutes: float) -> datetime:
return datetime.now(UTC) - timedelta(minutes=minutes)
+203
View File
@@ -0,0 +1,203 @@
import asyncio
import contextlib
from collections.abc import AsyncIterator
from datetime import UTC, datetime, timedelta
import httpx
import pytest
from app.db.models import ContainerSnapshot
from app.db.session import SessionMaker
from app.runtime import checks
from app.runtime.checks import Checker, CheckName, ChecksConfig, Verdict, count_errors, load_checks
from app.runtime.docker import DockerClient, Light
from app.runtime.snapshots import SnapshotStore, collect_once, snapshot_loop
from tests.conftest import ROOT
from tests.runtime.fake_docker import FakeDocker, now_minus
CFG = load_checks(ROOT / "config")
def probe(request: httpx.Request) -> httpx.Response:
if request.url.host == "broken":
raise httpx.ConnectError("refused", request=request)
return httpx.Response(200 if request.url.path == "/health" else 503)
@pytest.fixture
def fake() -> FakeDocker:
fake = FakeDocker()
fake.add("hello", service="hello", labels={"tgit.health_url": "http://hello:8000/health"})
fake.add(
"unhealthy", service="unhealthy", health="unhealthy", labels={"tgit.health_url": "http://unhealthy:8000/x"}
)
fake.add("crashed", state="exited", health=None, exit_code=137, oom=True)
return fake
@pytest.fixture
async def docker(fake: FakeDocker) -> AsyncIterator[DockerClient]:
async with fake.client() as client:
await client.containers() # the bot always lists before it inspects
yield client
@pytest.fixture
def store(sessions: SessionMaker) -> SnapshotStore:
return SnapshotStore(sessions)
@pytest.fixture
def checker(docker: DockerClient, store: SnapshotStore) -> Checker:
return Checker(docker, store, CFG, httpx.AsyncClient(transport=httpx.MockTransport(probe)))
def verdicts(report: checks.ContainerReport) -> dict[CheckName, Verdict]:
return {r.name: r.verdict for r in report.results}
async def test_config_loads_and_bad_pattern_rejected() -> None:
assert CFG.window == 300
with pytest.raises(ValueError):
ChecksConfig(error_pattern="(unclosed")
async def test_healthy_vs_unhealthy_vs_crashed(checker: Checker) -> None:
reports = {r.details.name: r for r in await checker.check_all()}
hello = verdicts(reports["hello"])
assert hello[CheckName.RUNNING] is Verdict.OK
assert hello[CheckName.HEALTH] is Verdict.OK
assert hello[CheckName.HTTP] is Verdict.OK
assert hello[CheckName.RESTARTS] is Verdict.SKIP # no history yet
assert reports["hello"].ok
bad = verdicts(reports["unhealthy"])
assert bad[CheckName.HEALTH] is Verdict.FAIL
assert bad[CheckName.HTTP] is Verdict.FAIL
assert not reports["unhealthy"].ok
crashed = reports["crashed"]
assert verdicts(crashed)[CheckName.RUNNING] is Verdict.FAIL
assert "exit 137, OOM" in crashed.results[0].detail
assert verdicts(crashed)[CheckName.CPU] is Verdict.SKIP
assert crashed.details.light is Light.RED
async def test_restarts_compared_with_history(
checker: Checker, fake: FakeDocker, docker: DockerClient, store: SnapshotStore
) -> None:
hello = next(c for c in fake.containers.values() if c.name == "hello")
assert await collect_once(docker, store, now=datetime.now(UTC) - timedelta(minutes=3)) == 2 # running ones only
hello.restart_count = 2
report = await checker.check(hello.id)
result = next(r for r in report.results if r.name is CheckName.RESTARTS)
assert result.verdict is Verdict.FAIL
assert result.detail == "0 → 2"
async def test_old_history_outside_window_is_ignored(
checker: Checker, fake: FakeDocker, sessions: SessionMaker
) -> None:
hello = next(c for c in fake.containers.values() if c.name == "hello")
async with sessions.begin() as s:
s.add(
ContainerSnapshot(container="hello", at=now_minus(30), restart_count=0, cpu_pct=1, mem_pct=1, health=None)
)
hello.restart_count = 5
report = await checker.check(hello.id)
assert verdicts(report)[CheckName.RESTARTS] is Verdict.SKIP
async def test_thresholds_and_log_errors(checker: Checker, fake: FakeDocker) -> None:
hello = next(c for c in fake.containers.values() if c.name == "hello")
hello.cpu_pct, hello.mem_bytes = 150.0, 250 * 1024 * 1024
hello.logs = [
(now_minus(20), "Traceback from long ago"),
(now_minus(1), "all good"),
(now_minus(1), '{"level": "ERROR", "msg": "boom"}'),
]
report = await checker.check(hello.id)
v = verdicts(report)
assert v[CheckName.CPU] is Verdict.FAIL
assert v[CheckName.MEM] is Verdict.FAIL
log_result = next(r for r in report.results if r.name is CheckName.LOG_ERRORS)
assert log_result.verdict is Verdict.FAIL
assert log_result.detail.startswith("1: ") # the old Traceback is outside the window
async def test_broken_or_slow_check_does_not_spoil_report(
checker: Checker, fake: FakeDocker, monkeypatch: pytest.MonkeyPatch
) -> None:
async def boom(ctx: checks.Context) -> checks.CheckResult:
raise RuntimeError("kaputt")
async def slow(ctx: checks.Context) -> checks.CheckResult:
await asyncio.sleep(10)
raise AssertionError
monkeypatch.setattr(checks, "CHECKS", [(CheckName.HEALTH, boom), (CheckName.HTTP, slow), *checks.CHECKS[:1]])
fast = Checker(checker._docker, checker._store, CFG.model_copy(update={"check_timeout": 0.05}), checker._http)
hello = next(c for c in fake.containers.values() if c.name == "hello")
report = await fast.check(hello.id)
by_name = {r.name: r for r in report.results}
assert by_name[CheckName.HEALTH].detail == "RuntimeError: kaputt"
assert by_name[CheckName.HTTP].detail == "timeout"
assert by_name[CheckName.RUNNING].verdict is Verdict.OK
async def test_stats_failure_skips_cpu_and_mem(checker: Checker, fake: FakeDocker) -> None:
hello = next(c for c in fake.containers.values() if c.name == "hello")
hello.stats_status = 500
report = await checker.check(hello.id)
assert verdicts(report)[CheckName.CPU] is Verdict.SKIP
assert verdicts(report)[CheckName.RUNNING] is Verdict.OK
async def test_http_probe_connection_error(fake: FakeDocker, checker: Checker) -> None:
hello = next(c for c in fake.containers.values() if c.name == "hello")
hello.labels["tgit.health_url"] = "http://broken/health"
report = await checker.check(hello.id)
result = next(r for r in report.results if r.name is CheckName.HTTP)
assert (result.verdict, result.detail) == (Verdict.FAIL, "ConnectError")
async def test_snapshot_store_latest(docker: DockerClient, store: SnapshotStore) -> None:
await collect_once(docker, store, now=now_minus(1))
await collect_once(docker, store)
latest = await store.latest("hello")
assert latest is not None and latest.cpu_pct == 5.0 and latest.health == "healthy"
def test_count_errors_reports_first_hit() -> None:
hits, first = count_errors("ok\nERROR one\nERROR two\n", r"ERROR")
assert (hits, first) == (2, "ERROR one")
async def test_snapshot_loop_survives_docker_outage(
fake: FakeDocker, docker: DockerClient, store: SnapshotStore
) -> None:
fake.down = True
task = asyncio.create_task(snapshot_loop(docker, store, interval=0.01))
await asyncio.sleep(0.05)
fake.down = False
await asyncio.sleep(0.05)
task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await task
assert await store.latest("hello") is not None
+110
View File
@@ -0,0 +1,110 @@
import pytest
from app.runtime.docker import DockerError, Light, demux_logs, light, parse_stats, parse_time
from tests.runtime.fake_docker import FakeDocker, MiB, frame, now_minus
@pytest.fixture
def fake() -> FakeDocker:
fake = FakeDocker()
fake.add(
"tgit-dev-bot-1", service="bot", labels={"tgit.sha": "526eb56aa", "tgit.deployed_at": "2026-10-17T07:52:00Z"}
)
fake.add("tgit-dev-postgres-1", service="postgres")
fake.add("vobot-api-1", managed=False) # someone else's project on the same host
return fake
async def test_lists_only_managed_containers(fake: FakeDocker) -> None:
async with fake.client() as docker:
names = [c.name for c in await docker.containers()]
assert names == ["tgit-dev-bot-1", "tgit-dev-postgres-1"]
assert all(r.method == "GET" for r in fake.requests)
async def test_foreign_container_is_refused_without_a_request(fake: FakeDocker) -> None:
foreign = next(c for c in fake.containers.values() if not c.managed)
async with fake.client() as docker:
await docker.containers()
fake.requests.clear()
with pytest.raises(DockerError):
await docker.inspect(foreign.id)
with pytest.raises(DockerError):
await docker.logs(foreign.id)
with pytest.raises(DockerError):
await docker.resolve(foreign.id[:12])
assert all("/containers/json" in str(r.url) for r in fake.requests) # only re-listings, never the foreign id
async def test_inspect_and_stats(fake: FakeDocker) -> None:
async with fake.client() as docker:
bot = next(c for c in await docker.containers() if c.service == "bot")
details = await docker.inspect(await docker.resolve(bot.short_id))
stats = await docker.stats(bot.id)
assert details.light is Light.GREEN
assert details.sha == "526eb56aa"
assert details.deployed_at is not None and details.deployed_at.hour == 7
assert details.started_at is not None and details.started_at.microsecond == 123456 # nanoseconds trimmed
assert stats.cpu_pct == 5.0
assert stats.mem_bytes == 64 * MiB # page cache subtracted
assert stats.mem_pct == 25.0
async def test_logs_are_demultiplexed_and_since_filtered(fake: FakeDocker) -> None:
bot = next(c for c in fake.containers.values() if c.service == "bot")
bot.logs = [(now_minus(10), "old line"), (now_minus(1), "fresh line"), (now_minus(0.5), "ERR on stderr")]
async with fake.client() as docker:
await docker.containers()
text = await docker.logs(bot.id, since=now_minus(5))
assert text == "fresh line\nERR on stderr\n"
async def test_unreachable_docker_is_a_docker_error(fake: FakeDocker) -> None:
fake.down = True
async with fake.client() as docker:
assert not await docker.ping()
with pytest.raises(DockerError, match="unreachable"):
await docker.containers()
async def test_ping(fake: FakeDocker) -> None:
async with fake.client() as docker:
assert await docker.ping()
def test_demux_plain_tty_stream_passes_through() -> None:
assert demux_logs(b"plain text\n") == "plain text\n"
assert demux_logs(frame("a\n") + frame("b\n", 2)) == "a\nb\n"
@pytest.mark.parametrize(
("state", "health", "code", "expected"),
[
("running", "healthy", 0, Light.GREEN),
("running", None, 0, Light.GREEN),
("running", "starting", 0, Light.YELLOW),
("running", "unhealthy", 0, Light.RED),
("restarting", None, 0, Light.RED),
("exited", None, 137, Light.RED),
("exited", None, 0, Light.WHITE),
("created", None, 0, Light.YELLOW),
],
)
def test_light(state: str, health: str | None, code: int, expected: Light) -> None:
assert light(state, health, code) is expected
def test_parse_time_edge_cases() -> None:
assert parse_time(None) is None
assert parse_time("0001-01-01T00:00:00Z") is None
assert parse_time("garbage") is None
def test_parse_stats_without_previous_sample() -> None:
stats = parse_stats({"cpu_stats": {"cpu_usage": {"total_usage": 5}}, "memory_stats": {}})
assert stats.cpu_pct == 0.0
assert stats.mem_pct == 0.0
View File
+167
View File
@@ -0,0 +1,167 @@
import json
from pathlib import Path
from typing import Any
import httpx
import pytest
import respx
from app.config import SourceCraftSettings
from app.sourcecraft.client import SourceCraftClient, SourceCraftError
from app.sourcecraft.models import EventType, RunStatus
FIXTURES = Path(__file__).resolve().parent.parent / "fixtures" / "sourcecraft"
API = "https://api.example.test"
REPO = "org/repo"
SHA = "932f4b1a144cfed4fed8eafd94433e0995e5b12d"
def fixture(name: str) -> Any:
return json.loads((FIXTURES / name).read_text(encoding="utf-8"))
class Sleeps:
def __init__(self) -> None:
self.delays: list[float] = []
async def __call__(self, delay: float) -> None:
self.delays.append(delay)
@pytest.fixture
def sleeps() -> Sleeps:
return Sleeps()
@pytest.fixture
async def client(sleeps: Sleeps) -> Any:
settings = SourceCraftSettings(token="secret", api_url=API, max_attempts=3) # type: ignore[arg-type]
async with SourceCraftClient(settings, sleep=sleeps) as c:
yield c
@respx.mock
async def test_list_runs_parses_real_response(client: SourceCraftClient) -> None:
route = respx.get(f"{API}/repos/{REPO}/cicd/runs").respond(json=fixture("runs.json"))
runs = await client.list_runs(REPO, page_size=3)
assert route.calls.last.request.headers["Authorization"] == "Bearer secret"
assert route.calls.last.request.url.params["page_size"] == "3"
assert [r.slug for r in runs] == ["1000002", "1000001", "1000000"]
assert runs[1].event_type is EventType.MANUAL
@respx.mock
async def test_failed_run_points_at_failed_cube(client: SourceCraftClient) -> None:
respx.get(f"{API}/repos/{REPO}/cicd/runs/1000001").respond(json=fixture("run_failed.json"))
run = await client.get_run(REPO, "1000001")
assert run.status is RunStatus.FAILED and run.status.finished
assert [(t, c.slug) for _, t, c in run.failed_cubes()] == [("check", "lint")]
assert all(not c.is_service for _, _, c in run.cubes())
assert run.dates.finished_at is not None
@respx.mock
async def test_unfinished_run_has_no_finish_date(client: SourceCraftClient) -> None:
respx.get(f"{API}/repos/{REPO}/cicd/runs/1000002").respond(json=fixture("run_processing.json"))
run = await client.get_run(REPO, "1000002")
assert not run.status.finished
assert run.dates.finished_at is None # API sends 1970-01-01 for "not yet"
check = run.task("check-and-deploy", "check")
assert check is not None and check.status is RunStatus.SUCCESS
assert run.task("check-and-deploy", "nope") is None
@respx.mock
async def test_commit_sha_from_meta_cube(client: SourceCraftClient) -> None:
respx.get(f"{API}/repos/{REPO}/cicd/runs/1000002").respond(json=fixture("run_processing.json"))
respx.get(f"{API}/repos/{REPO}/cicd/logs/1000002/check-and-deploy/check/meta").respond(
json=fixture("log_meta.json")
)
run = await client.get_run(REPO, "1000002")
assert await client.run_commit_sha(REPO, run) == SHA
@respx.mock
async def test_commit_sha_none_when_meta_log_missing(client: SourceCraftClient) -> None:
respx.get(f"{API}/repos/{REPO}/cicd/runs/1000002").respond(json=fixture("run_processing.json"))
respx.get(f"{API}/repos/{REPO}/cicd/logs/1000002/check-and-deploy/check/meta").respond(
404, json={"error_code": "NotFound", "message": "no logs"}
)
run = await client.get_run(REPO, "1000002")
assert await client.run_commit_sha(REPO, run) is None
@respx.mock
async def test_logs_are_read_page_by_page(client: SourceCraftClient) -> None:
url = f"{API}/repos/{REPO}/cicd/logs/1/wf/task/cube"
respx.get(url, params={"page": "1"}).respond(json={"logs": "a\n", "page_complete": True, "done": False})
respx.get(url, params={"page": "2"}).respond(json={"logs": "b\n", "page_complete": True, "done": True})
respx.get(url, params={"page": "3"}).respond(404, json={"error_code": "NotFound", "message": ""})
logs = await client.get_cube_logs(REPO, "1", "wf", "task", "cube")
assert logs.text == "a\nb\n" and logs.done
@respx.mock
async def test_real_failed_lint_log(client: SourceCraftClient) -> None:
respx.get(f"{API}/repos/{REPO}/cicd/logs/1000001/check-and-deploy/check/lint").respond(
json=fixture("log_lint_failed.json")
)
logs = await client.get_cube_logs(REPO, "1000001", "check-and-deploy", "check", "lint")
assert "F821 Undefined name" in logs.text and logs.done
@respx.mock
async def test_retries_transport_errors_and_5xx(client: SourceCraftClient, sleeps: Sleeps) -> None:
respx.get(f"{API}/repos/{REPO}/cicd/runs").mock(
side_effect=[httpx.ConnectError("tls eof"), httpx.Response(503), httpx.Response(200, json={"runs": []})]
)
assert await client.list_runs(REPO) == []
assert sleeps.delays == [0.5, 1.0]
@respx.mock
async def test_gives_up_after_max_attempts(client: SourceCraftClient) -> None:
respx.get(f"{API}/repos/{REPO}/cicd/runs").mock(side_effect=httpx.ConnectError("down"))
with pytest.raises(httpx.ConnectError):
await client.list_runs(REPO)
@respx.mock
async def test_persistent_5xx_raises_api_error(client: SourceCraftClient) -> None:
respx.get(f"{API}/repos/{REPO}/cicd/runs").respond(502, text="bad gateway")
with pytest.raises(SourceCraftError) as exc:
await client.list_runs(REPO)
assert exc.value.status == 502
@respx.mock
async def test_4xx_is_not_retried(client: SourceCraftClient, sleeps: Sleeps) -> None:
route = respx.get(f"{API}/repos/{REPO}/cicd/runs/9").respond(
403, json={"error_code": "PermissionDenied", "message": "no access"}
)
with pytest.raises(SourceCraftError) as exc:
await client.get_run(REPO, "9")
assert (exc.value.status, exc.value.code) == (403, "PermissionDenied")
assert route.call_count == 1 and sleeps.delays == []
async def test_run_url(client: SourceCraftClient) -> None:
assert client.run_url(REPO, "7") == "https://sourcecraft.dev/org/repo/cicd/runs/7"
+3 -2
View File
@@ -23,8 +23,9 @@ def test_age_decides(tmp_path: Path) -> None:
async def test_heartbeat_touches_file(tmp_path: Path) -> None:
hb = tmp_path / "hb"
task = asyncio.create_task(health.heartbeat(hb, interval=10))
await asyncio.sleep(0)
assert hb.exists()
async with asyncio.timeout(2):
while not hb.exists(): # noqa: ASYNC110 - polling the filesystem; touch runs in a worker thread
await asyncio.sleep(0.01)
task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await task
Generated
+35
View File
@@ -74,6 +74,19 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/68/30/173960c42b05a6c59f7558e4b12a4b0d9ba376cf6aa9bde7f9e08a30ca8d/aiohttp-3.14.5-py3-none-any.whl", hash = "sha256:efc21a454892828368b11c2c780de0ff8bc991f73f6b99c6b66e56205470929b", size = 279517, upload-time = "2026-10-11T01:05:08.523Z" },
]
[[package]]
name = "aiohttp-socks"
version = "0.12.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohttp" },
{ name = "python-socks" },
]
sdist = { url = "https://files.pythonhosted.org/packages/18/1d/a306e0111222180e60f17131a3f5d9bc694dd999a8115959a7dd76c2238e/aiohttp_socks-0.12.0.tar.gz", hash = "sha256:3caf9f5a4164611122d412bc11b2f9114fd29c85e1ba27bb38060d3c236bdc8d", size = 12061, upload-time = "2026-08-12T04:43:15.791Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/86/64/ca6289632020523ea1841f01363f83ada10eeb0f21dd931fc1118dd85668/aiohttp_socks-0.12.0-py3-none-any.whl", hash = "sha256:ba6f95ec775c761d87f8578ab48f137d0457c676da104984202bf75e747d5ee6", size = 10693, upload-time = "2026-08-12T04:43:14.524Z" },
]
[[package]]
name = "aiosignal"
version = "1.4.0"
@@ -642,6 +655,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/60/d1/38f3a3405989a89ac18390803e70c6ad7c7760da4f9b83cbeca0c44a0c72/python_dotenv-1.2.4-py3-none-any.whl", hash = "sha256:42269a8a5b3fd54ffa6f3d84b18abed50064717576b4ecf03dc4a55d8aa04fdc", size = 23266, upload-time = "2026-10-01T05:36:08.633Z" },
]
[[package]]
name = "python-socks"
version = "3.1.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/04/ad/484ffb79532517b11a90af38647c38652224650b31a7ae1cedd5a418d8ab/python_socks-3.1.1.tar.gz", hash = "sha256:8d3e817cdbe858dc0bb8c8fdc8e79b6ce37acce110d33374c6f57a675cc9029e", size = 232781, upload-time = "2026-09-08T13:03:31.058Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/3b/23/2c2cef1b4313c55d1713201acd4ee2043fb2cf22e546ca9d36bf4317faea/python_socks-3.1.1-py3-none-any.whl", hash = "sha256:327e0d6378702c73a7790bf732e9f01392f17b48c7348a50b5bd1f710c2df1be", size = 49566, upload-time = "2026-09-08T13:03:29.604Z" },
]
[[package]]
name = "pyyaml"
version = "6.0.3"
@@ -763,6 +785,7 @@ version = "0.1.0"
source = { editable = "." }
dependencies = [
{ name = "aiogram" },
{ name = "aiohttp-socks" },
{ name = "alembic" },
{ name = "asyncpg" },
{ name = "httpx" },
@@ -784,11 +807,13 @@ dev = [
{ name = "respx" },
{ name = "ruff" },
{ name = "types-pyyaml" },
{ name = "types-regex" },
]
[package.metadata]
requires-dist = [
{ name = "aiogram", specifier = ">=3.13" },
{ name = "aiohttp-socks", specifier = ">=0.9" },
{ name = "alembic", specifier = ">=1.13" },
{ name = "asyncpg", specifier = ">=0.29" },
{ name = "httpx", specifier = ">=0.27" },
@@ -810,6 +835,7 @@ dev = [
{ name = "respx", specifier = ">=0.21" },
{ name = "ruff", specifier = ">=0.6" },
{ name = "types-pyyaml", specifier = ">=6.0" },
{ name = "types-regex", specifier = ">=2026.9.29.20261017" },
]
[[package]]
@@ -821,6 +847,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/15/c0/fc0644b7ddcfb969e95845837143cb5173ddd6e06ee4ba5fc493cd9329b7/types_pyyaml-6.0.12.20260906-py3-none-any.whl", hash = "sha256:bca893ff0d51df5c9053137d5d0e6ccd36e939a196356f1d5c16372422f5137b", size = 21282, upload-time = "2026-09-06T06:35:34.372Z" },
]
[[package]]
name = "types-regex"
version = "2026.9.29.20261017"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/a2/bb/15d3c5f2f21ebf040430970e57bf8c438701e21ecd155607860e1dcff4f2/types_regex-2026.9.29.20261017.tar.gz", hash = "sha256:1eb0d21f77331e0f87268eeb1a75c4ac8025665a211c6e2e8ba084504b1c299f", size = 13971, upload-time = "2026-10-17T06:20:04.659Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/bd/2d/4a7cde6131caf872b33d25a66ffd938ebd10758e84527eb3c456cb558e77/types_regex-2026.9.29.20261017-py3-none-any.whl", hash = "sha256:d0bc5d8f44d2039f40f52b272e672266ba6cc1710956c6c7f5045000fea433b8", size = 11194, upload-time = "2026-10-17T06:20:03.623Z" },
]
[[package]]
name = "typing-extensions"
version = "4.16.0"