[CI] API Tests красный 3 рана подряд: agent-instructions-routes «returns bundle metadata» — timeout 5s #170
Closed
opened 2026-07-04 15:21:24 +00:00 by andrei
·
5 comments
No Branch/Tag specified
master
ceo/pix-11114-gitnexus-metadata
ceo/pix-15506-api-write-500-fix
ceo/pix-15506-api-500-fix
ceo/pix-15497-dependency-security-updates
fix/pix-15500-lockfile-policy
chore/sync-master-ahead-pix-15377
fix/heartbeat-review-wait-retry-scheduling
agent/devops/pix-15345
chore/sync-master-ahead-pix-15193
fix/pix-15110-sentry-production-projects
fix/quota-window-health-json-output
agent/fullstack/pix-14770
agent/fullstack/pix-14729
agent/devops/pix-14664
fix/pm2-waiting-restart-watchdog
upgrade/paperclip-v2026.720.0
agent/devops/pix-14323
fix/sentry-autonomous-incident-routing
fix/sentry-canonical-stack-gate
fix/pix-11114-suppress-loop-success
chore/pix-11114-gitnexus-metadata
agent/devops/pix-14274
pix-14255-auto-prompt-check-speed
agent/cto/pix-14252-codex-config
agent/fullstack/pix-13817
agent/devops/pix-14192
agent/devops/pix-14184-quota-windows-health
fix/watchdog-ceo-triggerdetail-system
fix/watchdog-wake-ceo-autofix-20260717
agent/fullstack/pix-13215
fix/pix-14170-timer-coalesces
fix/heartbeat-watchdog-interval-aware-20260717
agent/devops/pix-13719
hotfix/paperclip-runtime-exports-20260708
fullstack/pix-13367-rebase
fix/shutdown-retry-chains
fix/server-vitest-config-root
fix/pix-14129-master-drift-delivery
fix/wake-prompt-payload-cap
fix/codex-worker-prompt-compaction
fix/sweeper-smoke-alert-guard-20260716
agent/cto/pix-14128-clean
agent/cto/pix-14128
agent/fullstack/pix-13959
fix/suppress-ceo-loop-no-action-20260716
fix/master-drift-terminal-run-cancel
agent/devops/pix-14069
agent/fullstack/pix-14084
agent/devops/pix-14027
agent/fullstack/pix-13996
agent/cto/pix-13983
fix/git-delivery-gate-costs-telemetry
agent/fullstack/pix-13907-pr304
agent/devops/pix-13910
fix/watchdog-ownership-timeout-delivery
fix/watchdog-ownership-timeout
agent/fullstack/pix-13894
agent/devops/pix-13864
agent/fullstack/pix-13777
agent/fullstack/pix-13762
agent/fullstack/pix-13661
fix/codex-retry-attempt-promotion
fix/suppress-git-delivery-handoff-20260715
agent/cto/pix-13838
agent/devops/pix-13841
fix/preserve-valid-queued-locks-20260715
fix/codex-auth-lock-wait-heartbeat-20260715
fix/control-plane-stale-execution-locks
agent/cto/pix-13774-forward-fix
fix/paperclip-pre-spawn-token-burn-20260715
agent/cto/pix-13791
fix/control-plane-watchdog-branch-drift
agent/cto/pix-13774
agent/cto/pix-13773-delivery
fix/scheduled-retry-watchdog-active-runs
ops/paperclip-auth-holds-active-adapters-20260714
agent/fullstack/pix-13767
fix/paperclip-forever-control-plane-guardrails-20260715
agent/devops/pix-13736
fix/paperclip-agent-handoff-report-hygiene-20260714
fix/codex-runtime-guard-master
fix/codex-output-monitor-stderr
agent/devops/pix-13720
ops/paperclip-autonomy-refill-20260714
agent/cto/pix-13695
ops/paperclip-runtime-stabilize-20260714
fix/codex-cli-global-flags
operator/pix-13685-inherited-claude-env
operator/pix-13671-control-plane-drift
operator/pix-13662-branch-drift
agent/fullstack/pix-13641
agent/devops/pix-13632
agent/fullstack/pix-13620
agent/devops/pix-13612
agent/cto/pix-13608
agent/fullstack/pix-13596
agent/devops/pix-13336
agent/ceo/pix-11114-forgejo-pr-files-fallback
agent/fullstack/pix-13497
agent/cto/pix-13525
agent/ceo/pix-11114-lightrag-policy-skip
agent/fullstack/pix-13521
agent/cto/pix-13349
fix/watchdog-pm2-hygiene-20260710
agent/fullstack/pix-13437
agent/cto/pix-13457
operator/control-plane-api-autorecover
agent/cto/pix-13391
agent/cto/pix-13423
agent/cto/pix-13419
agent/cto/pix-13388
operator/model-profile-schema-fix
agent/devops/pix-13422
agent/devops/pix-13361
agent/devops/pix-13396-sync
agent/cto/pix-13383
agent/fullstack/pix-13355
agent/devops/pix-13357-gitignore-agent-manifest
agent/fullstack/pix-13338
agent/fullstack/pix-13328
agent/devops/pix-13334
agent/cto/pix-13326
agent/cto/pix-13273
agent/fullstack/pix-13312
agent/devops/pix-13299
agent/cto/pix-13272
agent/devops/pix-13286
agent/fullstack/pix-13268
agent/cto/pix-13257
agent/fullstack/pix-13264
agent/fullstack/pix-13260
agent/fullstack/pix-13247
agent/devops/pix-13236
recovery/pix-13213-heartbeat-fixes
agent/devops/pix-13213-delivery
delivery/pix-12637
fix/approval-answer-label-fix
delivery/pix-13034
delivery/pix-13142
agent/devops/pix-13081-guard-delivery
agent/fullstack/pix-13186
agent/fullstack/pix-13174
agent/fullstack/pix-13144
agent/fullstack/pix-13145
agent/devops/pix-13136
agent/fullstack/pix-13130
feature/claude-issue-comment-wakeup-unhandled-rejection
agent/devops/pix-13125
feature/claude-fix-flaky-issue-list-limit
agent/devops/pix-13092
agent/devops/pix-13081
agent/cto/pix-13064
agent/fullstack/pix-13045
agent/cto/pix-13590-fix-master-ci-issue-list-mocks
agent/fullstack/pix-13034
agent/fullstack/pix-13019
agent/cto/pix-13030
agent/fullstack/pix-13011
agent/cto/pix-12986-terminal-disposition-fix
merge-queue/pix-13034-ci-api-job
merge/pix-12937-batch2
agent/devops/pix-12872
agent/cto/pix-12884-cherry-pick-forgejo-sweeper
fix/pix-12885-pr-gates-issue-scope
agent/cto/pix-12887
agent/europatech-hermes-agent/pix-11114
agent/devops/pix-12840
agent/devops/approvalbot-compound-issue
agent/cto/pix-12778
agent/devops/pix-12774
fix/issues-execution-state-read-response
fix/pix-12752-timer-heartbeat-checked
agent/cto/pix-12749
agent/devops/pix-13081-guard
agent/cto/pix-12726
agent/cto/pix-12674-heartbeat-shutdown-grace
agent/fullstack/pix-12684
agent/devops/pix-12641
agent/devops/pix-13883
agent/devops/pix-13781
fix/cl-watchdog-flock-pathfix
agent/devops/pix-13218
fix/claude-watchdog-argparse
agent/devops/pix-13138
fix/claude-daemon-audit
agent/fullstack/pix-13102
fix/telegram-interaction-replies
feature/claude-agents-lockfile-sync
fix/ci-lockfile-mismatch
agent/devops/pix-13039
agent/fullstack/pix-13055
agent/fullstack/pix-12998
agent/fullstack/pix-12993
agent/fullstack/pix-12985
fix/pagination-response-format
fix/pix-12761-cancel-done-runs
review-pix12826
agent/fullstack/pix-12826
fix/pix-12815-bounded-workspace-api
fix/heartbeat-workspace-policy
agent/devops/pix-12759
agent/cto/pix-12761
agent/fullstack/pix-12765
fix/pix-12751-done-target-live-run
agent/devops/pix-12740
agent/fullstack/pix-12708
fix/pix-12732-terminal-live-runs
agent/fullstack/pix-12682
agent/devops/pix-12667
agent/devops/pix-10781
pix-10781-fix-merged
agent/devops/pix-11070
agent/cto/pix-12560
agent/fullstack/pix-12103
agent/devops/pix-11100
pix-11070-remote
codex/paperclip-memory-system-loopback-proxy-recovered
agent/fullstack/pix-12212
agent/fullstack/pix-12181
agent/fullstack/pix-12095
agent/cto/pix-12065
fix/agents-lockfile-overrides-sync
agent/cto/pix-11803
agent/fullstack/pix-11638
agent/fullstack/pix-11596
agent/sentry/pix-11612
agent/sentry/pix-11604
agent/sentry/pix-11581
agent/devops/pix-11564
agent/fullstack/pix-11528
agent/fullstack/pix-11433
agent/devops/pix-11396
agent/devops/pix-11376
agent/fullstack/pix-11294
agent/devops/pix-10908
fix/PIX-7666-codex-flags-v2
agent/devops/pix-10782
agent/devops/pix-10772
fix/pix-10731-workspace-closed-consistency
agent/devops/pix-10734
agent/devops/pix-10721
agent/devops/pix-11143
agent/cto/pix-12251
agent/cto/pix-12286
agent/cto/pix-12433
agent/cto/pix-12587
agent/cto/pix-12897
agent/cto/pix-12909
agent/cto/pix-13478
agent/devops/pix-12910
agent/devops/pix-12914
agent/devops/pix-12941
agent/devops/pix-12942
agent/devops/pix-12954
agent/devops/pix-12957
agent/devops/pix-12961
agent/devops/pix-12962
agent/devops/pix-12980
agent/devops/pix-12984
agent/devops/pix-12988
agent/devops/pix-12991
agent/devops/pix-13010
agent/devops/pix-13108
agent/devops/pix-13123
agent/devops/pix-13164
agent/devops/pix-13167
agent/devops/pix-13175
agent/devops/pix-13177
agent/devops/pix-13182
agent/devops/pix-13188
agent/devops/pix-13193
agent/devops/pix-13196
agent/devops/pix-13252
agent/devops/pix-13267
agent/devops/pix-13274
agent/devops/pix-13280
agent/devops/pix-13288
agent/devops/pix-13295
agent/devops/pix-13300
agent/devops/pix-13306
agent/devops/pix-13308
agent/devops/pix-13312
agent/devops/pix-13362
agent/devops/pix-13369
agent/devops/pix-13385
agent/devops/pix-13417
agent/devops/pix-13436
agent/devops/pix-13439
agent/devops/pix-13450
agent/devops/pix-13452
agent/devops/pix-13471
agent/fullstack/pix-12313
agent/fullstack/pix-12651
deploy/paperclip-v2026.618.0-20260619
fix/PIX-7823-v2-local
fix/PIX-7823-v2
fix/pix-7555-benign-cancel-full
feature/claude-recovery-spread
reconcile/prod-sync-20260615
feature/claude-stranded-recovery-autoclose
fix/PIX-8194-productivity-delivery-signal-015229
fix/run-liveness-terminal-cancelled
fix/PIX-7723-v2-git-status-gate
fix/PIX-8262-transient-retry-escalation
fix/PIX-8157-transient-productivity-review
fix/PIX-8111-scheduled-retry-blocker
fix/PIX-7723-done-gate-non-code-sla
fix/PIX-7823-budget-guardrail-normalization
fix/PIX-7814-regulatory-template-claims
fix/PIX-7831-queued-retry-drain
fix/PIX-7733-cost-attribution
fix/PIX-7595-costs-by-agent-usage
fix/PIX-7666-codex-recovery-flags-clean
fix/PIX-7714-lightrag-maintenance
fix/pix-7662-metered-codex-cost
fix/pix-7684-superseded-done-gate-clean
fix/pix-7684-superseded-done-gate
fix/PIX-7666-codex-recovery-flags
fix/pix-7473-costs-current-month
fix/pix-7560-support-sla-done-gate
fix/pix-7632-codex-search-fallback
fix/pix-7555-benign-cancelled-snapshots
fix/pix-7573-cfo-routine-gate
fix/pix-7496-forgejo-ci-devdeps
fix/pix-7497-code-task-gate
fix/agent-deliverable-doc-tg
fix/pix-7437-recovery-blocked-path
fix/pix-7245-codex-search-fallback
agent-cto/pix-7481-git-delivery-gate
fix/pix-7435-research-closeout-gate
fix/pix-7433-budget-gate
agent-cto/pix-7374-agents
fix/pix-7277-git-delivery-fail-closed
feature/claude-pix-7086-impl
feature/claude-pix-7086-done-gate
chore/reconcile-source-drift
archive/master-pre-reconcile-52b4555
europa-v2
cleanup/codex-git-junk-20260424-035731
refactor/ns-modules
europa
feature/observability-dashboard
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
europa-tech-srl/europa-tech-agents#170
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Стабильный фейл (не флейк): task 35358 (
701fd241), 35384 (f46f5d23), 35411 (2fea87b8).Из лога 35384:
Тест ждёт GET /api/agents//instructions-bundle с mockAgentInstructionsService.getBundle — таймаут указывает, что роут не доходит до мока (висящий await в новом код-пути?). Началось в окне merge #168 и соседних коммитов.
Диагностика снята с runner-логов Forgejo (/data/gitea/actions_log/.../35384.log.zst). Локальная Windows-репродукция невозможна (embedded-postgres). Чинить владельцам репо на 178.
Статус после PR #169 (merge
32b6f8b6, 2026-07-04):Закрыто в PR #169:
europatech-postgresна раннере. Убран publish,DATABASE_URLчерез docker alias.routes/agents.tsвнутри 5s бюджета. ЗакрытоtestTimeout: 30_000+ beforeAll-прогрев в agent-instructions.agent-live-run-routes(companyId UUID,{summary:false}),approval-routes-idempotency(UUID-фикстуры подisUuidLike).companies.tsпотеряны при upstream-merge v2026.626.0 — восстановлены из прод-снапшота247c7749.Остаток (для PIX-13034 cleanup):
adapter-routes-authz«allows instance admins to reach mutating adapter routes»:npm install failed: spawn npm ENOENT— только в CI (node:22 контейнер), локально 4/4 зелёный. Мокnode:child_process.execFileв CI-среде не перехватывает вызов — разобрать env-разницу (Node 22 vs 24 / vitest builtin mocking).git showдвух ревизий»: тест устарел или код потерял форк-патч.Джоба остаётся non-blocking (
continue-on-error) до конца cleanup — задумано, run-статус при этом красный из-за известного ограничения forgejo-runner.Дополнение после глубокого разбора (без Linux-репро):
5fdd55b78(страница 2 истории тасков) — не регрессия #168. Похоже, BP не требует этот джоб и мержится мимо.Рекомендация: воспроизвести под Linux одним vitest-файлом, проверить пункт 3; фикс, скорее всего, — выравнивание харнеса (использовать top-level vi.mock без resetModules-цикла, как в соседних route-тестах).
РЕПРО-ПАК (стенд /tmp/agents-fix на 178, логи /tmp/af-*.log):
ВЫВОД: межфайловая утечка НЕ через module-graph (форки изолируют), а через ВНЕШНЕЕ разделяемое состояние. Кандидаты: (а) реальные пути /tmp/agent-1 и т.п. в фикстурах — если какой-то тест материализует их на ФС, соседи начинают видеть реальные каталоги; (б) singleton embedded-postgres/фоновые таймеры, переживающие файл; (в) env-мутации.
Следующий шаг владельцу (1 команда на стенде): прогнать serialized ДВАЖДЫ подряд и сравнить падения; затем ls /tmp/agent-* после прогона — если каталоги существуют, виновник — тест, пишущий фикстуры на реальную ФС без cleanup.
Диагност: DevSecOps Core. Стенд оставлен в /tmp/agents-fix (можно сносить).
Слои 6-7 закрыты PR #174 (merge). Следующий слой (run 35650):
issue-agent-mutation-ownership-routes— 11/62 failed, воспроизводится локально на чистом master (не CI-env). Три класса: (1) issue list fast path отвечает 500 Internal; (2) требованиеAgent-created issues must include an explicit projectIdпротив старых фикстур; (3)invalid_issue_dispositionвалидация против agent-authored updates. Классы 2-3 похожи на новые валидации из партии PIX-12986/PIX-12937 без обновления тестов; класс 1 требует разбора stack trace. Метод прежний: git show двух ревизий по каждому классу.Закрыто. Master push-ран
c3982b61:Agents CI / API Tests (push) = success,Typecheck and Build (push) = success— первый полностью зелёный master за партию.Серия PR: #169 (порт-коллизия postgres + transform-класс + лимиты джобы), #174 (annotation-wake drift + tombstone DELETE), #175 (security: visibility-фильтр списка, createdByRunId guard, watchdog-сериализация), #177 (attachment byte-range + canonicalize), #178 (снятие 10s overrides + telemetry-моки), #180 (closed-workspace моки), #181 (list-limit пагинация/blockedBy), #182 (watchdog reconcile мок), #184 (clear-error роут восстановлен, OpenAPI-контракт доведён, cross-tenant suite раскван).
Итог по типам: 6 потерянных при merge v2026.626.0 маршрутов/guard-ов восстановлены (в т.ч. 2 security), 1 фантомная spec-запись закрыта восстановлением кода, остальное — тесты/моки, отставшие от живых форк-фич. Метод для будущих merge: «мёртвое определение» (helper/spec-запись без вызовов) — самый быстрый детектор потерь.