Skip to content

[MTA-6525] Conceptual overview of Solution Server and LLM Integration - #395

Open
mpershina wants to merge 1 commit into
mainfrom
MTA-6525
Open

[MTA-6525] Conceptual overview of Solution Server and LLM Integration#395
mpershina wants to merge 1 commit into
mainfrom
MTA-6525

Conversation

@mpershina

@mpershina mpershina commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Tracked under MTA-6525.

Preview: https://mpershina.github.io/Previews/MTA/SS-LLM-integration.html#solution-server_dl-context

Summary by CodeRabbit

  • Documentation
    • Added documentation on Solution Server and large language model integration.
    • Explained how AI-assisted code suggestions, retrieval-augmented context, and iterative resolution work.
    • Documented security practices for handling API credentials.
    • Updated Agent AI and Solution Server content to reflect local code fixes and stored migration insights.
    • Added the new integration section to the Developer Lightspeed documentation.

@mpershina mpershina self-assigned this Jul 29, 2026
@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Changes

Developer Lightspeed AI documentation

Layer / File(s) Summary
Solution Server and LLM integration flow
docs/topics/developer-lightspeed/con_solution-server-and-large-language-model-integration.adoc, docs/topics/developer-lightspeed/assembly_understanding_dl.adoc
Adds a topic covering LLM collaboration, RAG context, agent mode, security handling, and includes it in the Developer Lightspeed assembly.
Agent AI and Solution Server behavior
docs/topics/developer-lightspeed/con_agent-ai.adoc, docs/topics/developer-lightspeed/con_solution_server.adoc
Reframes Agent AI around local fixes, documents stored migration resolutions and confidence metrics, and removes the configurable LLM provider table.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Sequence Diagram(s)

sequenceDiagram
  participant IDEExtension
  participant SolutionServer
  participant Agent
  participant LLM
  IDEExtension->>SolutionServer: request migration insights
  SolutionServer->>Agent: provide resolution context
  Agent->>LLM: request code suggestions
  LLM-->>Agent: return suggested resolution
  Agent-->>IDEExtension: present local changes for review
Loading

Possibly related PRs

Suggested reviewers: anarnold97

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: a conceptual overview of Solution Server and LLM integration documentation.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch MTA-6525

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/topics/developer-lightspeed/con_agent-ai.adoc`:
- Around line 12-16: Keep the Agent AI approval boundary explicit: in
docs/topics/developer-lightspeed/con_agent-ai.adoc lines 12-16, describe
generated local suggestions as requiring user review and approval before
modifications are applied; in
docs/topics/developer-lightspeed/con_solution-server-and-large-language-model-integration.adoc
line 23, clarify that agent automation does not remove this review step.

In
`@docs/topics/developer-lightspeed/con_solution-server-and-large-language-model-integration.adoc`:
- Line 13: Update the Solution Server description to state only the documented
behavior: accepted resolutions generate solved examples and migration hints that
inform future prompts. Remove or qualify the claim that rejected code changes
are evaluated, ingested, or stored unless that behavior is explicitly documented
elsewhere.
- Line 13: Update the Solution Server flow description and the corresponding
references at the other indicated occurrences to remove direct-to-LLM wording.
State that requests are proxied through the LLM proxy and credentials or secrets
remain managed within the cluster and are consumed server-side; keep the
surrounding integration behavior unchanged.
- Line 15: Update the RAG description in the developer Lightspeed integration
documentation to state that RAG retrieves and augments context, while the LLM
uses that context to generate issue resolutions. Rephrase the sentence without
implying that RAG itself produces code fixes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e5eac0cd-8be7-4167-97a3-fafd59a0b3cc

📥 Commits

Reviewing files that changed from the base of the PR and between 4ad34f6 and 6f44bb0.

📒 Files selected for processing (4)
  • docs/topics/developer-lightspeed/assembly_understanding_dl.adoc
  • docs/topics/developer-lightspeed/con_agent-ai.adoc
  • docs/topics/developer-lightspeed/con_solution-server-and-large-language-model-integration.adoc
  • docs/topics/developer-lightspeed/con_solution_server.adoc

Comment on lines +12 to +16
To understand client-side autonomous execution, review how the Agent AI operates within your environment. The Agent AI performs local code modifications directly in your workspace.

* Plans the context to define the issues.
* Chooses a suitable sub agent for the analysis task. {mta-dl-plugin} works with the LLM to generate fix suggestions. The LLM displays the reasoning transcript and files to be changed.
* Applies the changes to the code once you approve the updates.
The Agent AI operates within the developer integrated development environment (IDE). It executes targeted fixes on specific files without requiring a full hub re-analysis cycle.

If you accept that the agentic AI must continue to make changes, it compiles the code and runs a partial analysis. In this iteration, the agentic AI attempts to fix diagnostic issues (if any) generated by tools that you installed in the Visual Studio Code IDE. You can review the changes and accept the agentic AI's suggestion to address these diagnostic issues.

After each iteration of applying changes to the code, the agentic AI asks if you want the agent to continue fixing more issues. When you accept, it runs another iteration of automated analysis until it has resolved all issues or it has made a maximum of two attempts to fix an issue.

Agentic AI generates a new preview in each iteration when it updates the code with the suggested resolutions. The time taken by the agentic AI to complete all iterations depends on the number of new diagnostic issues that are detected in the code.
When diagnostic or migration issues are discovered in a file, the Agent AI generates local resolution suggestions. It presents proposed changes directly within the workspace for user review before applying them.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Keep the Agent AI approval boundary consistent.

The two descriptions can imply automatic application of fixes, while the documented workflow requires users to review proposed changes before they are applied.

  • docs/topics/developer-lightspeed/con_agent-ai.adoc#L12-L16: describe local suggestions and user-approved modifications.
  • docs/topics/developer-lightspeed/con_solution-server-and-large-language-model-integration.adoc#L23-L23: clarify that agent automation does not remove the review step.
📍 Affects 2 files
  • docs/topics/developer-lightspeed/con_agent-ai.adoc#L12-L16 (this comment)
  • docs/topics/developer-lightspeed/con_solution-server-and-large-language-model-integration.adoc#L23-L23
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/topics/developer-lightspeed/con_agent-ai.adoc` around lines 12 - 16,
Keep the Agent AI approval boundary explicit: in
docs/topics/developer-lightspeed/con_agent-ai.adoc lines 12-16, describe
generated local suggestions as requiring user review and approval before
modifications are applied; in
docs/topics/developer-lightspeed/con_solution-server-and-large-language-model-integration.adoc
line 23, clarify that agent automation does not remove this review step.


== Collaborative workflow

The Solution Server is an independent backend component that builds a collective memory of your migration projects. Your editor extension communicates directly with the large language model (LLM) or routes requests through an LLM proxy. The Solution Server evaluates your accepted and rejected code changes to extract additional insights and resolution patterns. It feeds these insights back to the extension to enhance the prompt context for future LLM solution requests.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Limit the Solution Server memory claim to documented behavior.

This says both accepted and rejected changes are evaluated, but the supplied behavior describes accepted resolutions creating solved examples and migration hints. Clarify whether rejected changes are actually ingested or stored, since this affects users’ expectations about what code reaches the backend.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@docs/topics/developer-lightspeed/con_solution-server-and-large-language-model-integration.adoc`
at line 13, Update the Solution Server description to state only the documented
behavior: accepted resolutions generate solved examples and migration hints that
inform future prompts. Remove or qualify the claim that rejected code changes
are evaluated, ingested, or stored unless that behavior is explicitly documented
elsewhere.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Describe the proxy and credential path consistently.

For the Solution Server flow, the supplied deployment contract requires the LLM proxy, which consumes the cluster-managed secret server-side. Remove the “directly to the LLM” wording and replace “credentials are routed through the proxy” with wording that explains requests are proxied while secrets remain in the cluster.

Also applies to: 19-19, 27-27

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@docs/topics/developer-lightspeed/con_solution-server-and-large-language-model-integration.adoc`
at line 13, Update the Solution Server flow description and the corresponding
references at the other indicated occurrences to remove direct-to-LLM wording.
State that requests are proxied through the LLM proxy and credentials or secrets
remain managed within the cluster and are consumed server-side; keep the
surrounding integration behavior unchanged.


The Solution Server is an independent backend component that builds a collective memory of your migration projects. Your editor extension communicates directly with the large language model (LLM) or routes requests through an LLM proxy. The Solution Server evaluates your accepted and rejected code changes to extract additional insights and resolution patterns. It feeds these insights back to the extension to enhance the prompt context for future LLM solution requests.

{mta-dl-plugin} uses Retrieval Augmented Generation (RAG). RAG produces context-based resolutions for issues in your code. By using RAG, the extension improves the context shared with the LLM.

@coderabbitai coderabbitai Bot Jul 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Attribute resolution generation to the LLM, not RAG.

RAG retrieves and augments context; the LLM uses that context to generate the resolution. Rephrase this sentence to avoid implying that RAG itself produces code fixes.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@docs/topics/developer-lightspeed/con_solution-server-and-large-language-model-integration.adoc`
at line 15, Update the RAG description in the developer Lightspeed integration
documentation to state that RAG retrieves and augments context, while the LLM
uses that context to generate issue resolutions. Rephrase the sentence without
implying that RAG itself produces code fixes.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{mta-dl-plugin} uses Retrieval Augmented Generation (RAG). RAG produces context-based resolutions for issues in your code. By using RAG, the extension improves the context shared with the LLM.
By using Retrieval Augmented Generation (RAG), {mta-dl-plugin} produces context-based code resolutions. The context improves as developers accept more migration fixes over time.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mpershina, this is closer, but it still leaves the LLM’s role implicit. To make the distinction unambiguous, I suggest:

By using Retrieval Augmented Generation (RAG), {mta-dl-plugin} retrieves and augments context for the LLM, which generates context-based code resolutions. The context improves as developers accept more migration fixes over time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant