feat(web): add optional PR/MR summary comment to review agent#1175
feat(web): add optional PR/MR summary comment to review agent#1175fatmcgav wants to merge 4 commits intosourcebot-dev:mainfrom
Conversation
Adds REVIEW_AGENT_SUMMARY_ENABLED (default false) and REVIEW_AGENT_SUMMARY_MAX_LENGTH (default 250) env vars. When enabled, the review agent generates a concise markdown summary of the PR/MR changes and posts it as a top-level comment before pushing inline review comments. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
WalkthroughAdds an experimental PR/MR summary feature to the Review Agent: two new env vars, a new summary-generation module that calls the AI text generator, wiring to conditionally produce a summary during PR/MR processing, and posting/updating summary comments on GitHub and GitLab. ChangesPR/MR Summary Generation Feature
Sequence DiagramsequenceDiagram
actor Agent as Review Agent
participant Env as Env Config
participant Gen as generatePrSummary
participant AI as Language Model
participant GH as GitHub API
participant GL as GitLab API
Agent->>Env: Check REVIEW_AGENT_SUMMARY_ENABLED
alt enabled
Agent->>Gen: generatePrSummary(prPayload)
Gen->>AI: select model, send prompts
AI-->>Gen: generated summary
Gen->>Agent: truncated summary
alt GitHub PR
Agent->>GH: githubPushPrReviews(..., summary)
GH-->>Agent: create/update issue comment
else GitLab MR
Agent->>GL: gitlabPushMrReviews(..., summary)
GL-->>Agent: create/edit MR note
end
else disabled
Agent->>Agent: skip summary generation
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
packages/web/src/features/agents/review-agent/app.ts (1)
60-67: ⚡ Quick winExtract duplicated summary generation into a helper.
Lines 60-67 and 85-92 are functionally identical aside from the log label ("PR" vs "MR"). Extracting a small helper keeps the two flows in sync if generation logic, error handling, or telemetry evolves later.
♻️ Proposed helper extraction
+async function generateSummaryIfEnabled( + prPayload: Parameters<typeof generatePrSummary>[0], + label: 'PR' | 'MR', +): Promise<string | undefined> { + if (!env.REVIEW_AGENT_SUMMARY_ENABLED) { + return undefined; + } + try { + return await generatePrSummary(prPayload); + } catch (error) { + logger.error(`Error generating ${label} summary: ${error}`); + return undefined; + } +}Then replace the duplicated blocks with:
- let summary: string | undefined; - if (env.REVIEW_AGENT_SUMMARY_ENABLED) { - try { - summary = await generatePrSummary(prPayload); - } catch (error) { - logger.error(`Error generating PR summary: ${error}`); - } - } + const summary = await generateSummaryIfEnabled(prPayload, 'PR');- let summary: string | undefined; - if (env.REVIEW_AGENT_SUMMARY_ENABLED) { - try { - summary = await generatePrSummary(prPayload); - } catch (error) { - logger.error(`Error generating MR summary: ${error}`); - } - } + const summary = await generateSummaryIfEnabled(prPayload, 'MR');Also applies to: 85-92
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/web/src/features/agents/review-agent/app.ts` around lines 60 - 67, Extract the duplicated PR/MR summary generation into a shared helper: create a function (e.g., generateSummarySafely or generatePrOrMrSummary) that accepts the payload and a label/tag ("PR" or "MR"), calls generatePrSummary(payload) inside a try/catch, returns the summary (or undefined) and logs errors using logger.error with the label included; then replace both duplicated blocks that set summary (the one using generatePrSummary(prPayload) and the MR equivalent at lines around 85-92) to call this new helper instead, preserving existing behavior and telemetry/error handling.packages/web/src/features/agents/review-agent/nodes/githubPushPrReviews.ts (1)
10-21: 🏗️ Heavy liftConsider idempotency to avoid duplicate summary comments on re-run.
Each invocation unconditionally creates a new top-level issue comment. With
REVIEW_AGENT_AUTO_REVIEW_ENABLED=true(or repeatedreviewcommands), the PR will accumulate one summary comment per run, which is more noticeable than for inline review comments since the summary is not anchored to a commit/line.Consider tagging the comment with a hidden marker (e.g.,
<!-- sourcebot-review-summary -->) and, on subsequent runs, listing existing issue comments to find andupdateCommentit instead of creating a new one. The same pattern would apply to the GitLab MR notes flow.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/web/src/features/agents/review-agent/nodes/githubPushPrReviews.ts` around lines 10 - 21, The current flow unconditionally calls octokit.rest.issues.createComment with summary (using pr_payload) which creates duplicate top-level comments on re-run; change this to be idempotent by first listing existing issue comments via octokit.rest.issues.listComments (using pr_payload.owner/repo/number), search for a comment containing a hidden marker like "<!-- sourcebot-review-summary -->", and if found call octokit.rest.issues.updateComment to replace its body, otherwise createComment including the marker; keep error handling via logger.error as is for both list/update/create operations.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@CHANGELOG.md`:
- Around line 8-11: The Unreleased changelog entry is missing a subsection
header; move the bullet "- [Experimental] Added support for generating a summary
comment to the AI Code review agent. [`#1175`]..." under a new "### Added"
sub-header within the "## [Unreleased]" section so the entry is properly
parented per guidelines (add "### Added" immediately beneath "## [Unreleased]"
and keep the existing bullet under that header).
---
Nitpick comments:
In `@packages/web/src/features/agents/review-agent/app.ts`:
- Around line 60-67: Extract the duplicated PR/MR summary generation into a
shared helper: create a function (e.g., generateSummarySafely or
generatePrOrMrSummary) that accepts the payload and a label/tag ("PR" or "MR"),
calls generatePrSummary(payload) inside a try/catch, returns the summary (or
undefined) and logs errors using logger.error with the label included; then
replace both duplicated blocks that set summary (the one using
generatePrSummary(prPayload) and the MR equivalent at lines around 85-92) to
call this new helper instead, preserving existing behavior and telemetry/error
handling.
In `@packages/web/src/features/agents/review-agent/nodes/githubPushPrReviews.ts`:
- Around line 10-21: The current flow unconditionally calls
octokit.rest.issues.createComment with summary (using pr_payload) which creates
duplicate top-level comments on re-run; change this to be idempotent by first
listing existing issue comments via octokit.rest.issues.listComments (using
pr_payload.owner/repo/number), search for a comment containing a hidden marker
like "<!-- sourcebot-review-summary -->", and if found call
octokit.rest.issues.updateComment to replace its body, otherwise createComment
including the marker; keep error handling via logger.error as is for both
list/update/create operations.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 54e03067-c720-40b9-8e33-a1cd66c4c79b
📒 Files selected for processing (10)
.devcontainer/docker-compose.yml.env.developmentCHANGELOG.mddocs/docs/configuration/environment-variables.mdxdocs/docs/features/agents/review-agent.mdxpackages/shared/src/env.server.tspackages/web/src/features/agents/review-agent/app.tspackages/web/src/features/agents/review-agent/nodes/generatePrSummary.tspackages/web/src/features/agents/review-agent/nodes/githubPushPrReviews.tspackages/web/src/features/agents/review-agent/nodes/gitlabPushMrReviews.ts
…marySafely helper Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ooter GitHub and GitLab summary posting now checks for an existing comment/note containing a hidden `<!-- sourcebot-review-summary -->` marker before writing. If found, the existing comment is updated in place (`updateComment` / `MergeRequestNotes.edit`); otherwise a new one is created, preventing duplicate summary comments on re-runs. A markdown footer is appended to every summary body showing "Created: <timestamp>" on first post and "Updated: <timestamp>" on subsequent runs, giving reviewers visibility into when the last review was performed. Tests: * mock helpers updated to include `issues.listComments/createComment/ updateComment`` and `MergeRequestNotes.all/edit`. * New test suites added for both GitHub and GitLab handlers covering: * no summary -> no API calls; * first post -> create with marker and "Created:" footer; * re-run -> update with "Updated:" footer and no duplicate create; * API failure -> no throw propagated. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/web/src/features/agents/review-agent/nodes/githubPushPrReviews.ts`:
- Around line 13-20: The code uses octokit.rest.issues.listComments and then
searches for SUMMARY_MARKER which only checks the first page of comments; update
the logic to fetch all comment pages (e.g. use
octokit.paginate(octokit.rest.issues.listComments, { owner: pr_payload.owner,
repo: pr_payload.repo, issue_number: pr_payload.number })) and then run the
existing = comments.find(...) check against the full aggregated list so you
reliably detect an existing SUMMARY_MARKER and avoid duplicate creates; keep the
rest of the body/ action logic (SUMMARY_MARKER, action, body) unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 7bdeb1d4-5362-422d-900a-267e87c014cb
📒 Files selected for processing (6)
CHANGELOG.mdpackages/web/src/features/agents/review-agent/app.tspackages/web/src/features/agents/review-agent/nodes/githubPushPrReviews.test.tspackages/web/src/features/agents/review-agent/nodes/githubPushPrReviews.tspackages/web/src/features/agents/review-agent/nodes/gitlabPushMrReviews.test.tspackages/web/src/features/agents/review-agent/nodes/gitlabPushMrReviews.ts
✅ Files skipped from review due to trivial changes (1)
- CHANGELOG.md
| const { data: comments } = await octokit.rest.issues.listComments({ | ||
| owner: pr_payload.owner, | ||
| repo: pr_payload.repo, | ||
| issue_number: pr_payload.number, | ||
| }); | ||
| const existing = comments.find(c => c.body?.includes(SUMMARY_MARKER)); | ||
| const action = existing ? "Updated" : "Created"; | ||
| const body = `${SUMMARY_MARKER}\n${summary}\n\n---\n*${action}: ${new Date().toUTCString()}*`; |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat -n packages/web/src/features/agents/review-agent/nodes/githubPushPrReviews.tsRepository: sourcebot-dev/sourcebot
Length of output: 3544
🌐 Web query:
octokit rest issues listComments pagination behavior
💡 Result:
For the Octokit REST endpoint that lists issue comments (GET /repos/{owner}/{repo}/issues/{issue_number}/comments), pagination works via GitHub’s standard REST pagination parameters (page, per_page) and the Link response header. Octokit’s pagination helpers follow that Link header. 1) REST endpoint (what Octokit is calling) - The endpoint supports per_page (max 100) and page, with defaults per_page=30 and page=1.[1] - Issue comments are ordered by ascending ID.[1] 2) How pagination is signaled (Link header) - When a response is paginated, GitHub includes a link header containing URLs for next/prev/first/last pages.[2] - If the endpoint does not support pagination or everything fits on one page, the link header is omitted.[2] 3) Octokit pagination behavior (what to expect in code) - Octokit’s REST pagination plugin (octokit.paginate from @octokit/plugin-paginate-rest) paginates “List …” endpoints by repeatedly issuing requests as long as a rel="next" URL is present in the response Link header.[3] - If the response lacks the Link header, octokit.paginate resolves with an array based on the single response.[3] 4) Practical pagination options - With the endpoint directly (octokit.rest.issues.listComments), set per_page and page yourself; per_page max is 100.[1] - With Octokit’s paginator, pass per_page=100 to reduce number of requests; it will continue automatically using rel="next" links.[3][1] 5) Small gotcha: streaming - An Octokit discussion indicates that streaming (custom Accept header / streaming assets) is not supported for listComments; use the normal JSON pagination approach instead.[4] Example usage (paginating all comments for an issue) - Iterate pages automatically: - const comments = await octokit.paginate(octokit.rest.issues.listComments, { owner, repo, issue_number, per_page: 100 }); (Relies on the Link header rel="next" mechanism described by the paginator plugin.[3])
Citations:
- 1: https://docs.github.com/en/rest/issues/comments
- 2: https://docs.github.com/rest/guides/traversing-with-pagination
- 3: https://github.com/octokit/plugin-paginate-rest.js/
- 4: How do I get the result of octokit.issues.listComments as a stream? octokit/octokit.js#2724
Fetch all issue-comment pages before deciding create vs. update.
octokit.rest.issues.listComments() returns only the first page (default 30 comments), so the existing lookup at line 18 can miss a summary comment on later pages. Use octokit.paginate(octokit.rest.issues.listComments, {...}) or manually iterate through pages to reliably find existing comments and avoid creating duplicates.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/web/src/features/agents/review-agent/nodes/githubPushPrReviews.ts`
around lines 13 - 20, The code uses octokit.rest.issues.listComments and then
searches for SUMMARY_MARKER which only checks the first page of comments; update
the logic to fetch all comment pages (e.g. use
octokit.paginate(octokit.rest.issues.listComments, { owner: pr_payload.owner,
repo: pr_payload.repo, issue_number: pr_payload.number })) and then run the
existing = comments.find(...) check against the full aggregated list so you
reliably detect an existing SUMMARY_MARKER and avoid duplicate creates; keep the
rest of the body/ action logic (SUMMARY_MARKER, action, body) unchanged.
1 participant
Summary
Adds an optional PR/MR summary comment to the AI Review Agent. When enabled, the agent generates a concise plain-text (with optional inline markdown) summary of the changes and posts it as a top-level comment on the PR/MR before pushing inline review comments.
REVIEW_AGENT_SUMMARY_ENABLEDenv var (default:false) to opt in to summary commentsREVIEW_AGENT_SUMMARY_MAX_LENGTHenv var (default:250) to control the maximum character length of the summarygeneratePrSummarynode that calls the configured LLM with the PR title, description, and list of changed files to produce the summary; the result is hard-capped atREVIEW_AGENT_SUMMARY_MAX_LENGTHcharactersoctokit.rest.issues.createCommentMergeRequestNotes.createreview-agent.mdxandenvironment-variables.mdx, and added to.env.developmentand.devcontainer/docker-compose.ymlCo-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
Summary by CodeRabbit
New Features
Documentation
Tests