AI Score Improvement & Fix Suggestions

CodeFrog’s Mega Report includes AI-powered analysis that runs entirely on-device using Apple Intelligence. It analyzes your report findings and generates prioritized improvement plans and fix suggestions — all without sending any data off your Mac.

Overview

The AI analysis feature has two modes:

AI Score Improvement Plan — Analyzes severity counts across all sections in your report. Identifies the worst-performing section, prioritizes which sections to fix first, and generates a comprehensive plan to improve your grade.
AI Fix Suggestions (per section) — Analyzes individual findings within a specific test section and explains each issue, how to fix it, and its severity impact.

Both modes process everything on-device via Apple’s Foundation Models framework. No data is sent to Apple, cloud services, or any third party.

Requirements

macOS 26 or later (Apple Intelligence requires macOS 26+)
Apple Intelligence enabled — Check System Settings > Apple Intelligence & Siri
A completed Mega Report with findings — the AI needs scan results to analyze
Not available on Windows or older macOS versions

How It Works

On-Device Processing

The feature uses Apple’s Foundation Models framework to run a language model directly on your Mac’s Neural Engine. The model is built into macOS 26 — no separate download, no API key, and no internet connection required.

What Data the AI Sees

The AI receives only minimal, structured data from your report:

Overall mode (Score Improvement Plan):

Your current grade letter (e.g., “B”)
Severity counts per section (e.g., “Security: critical 1, high 3, medium 7”)
Which sections were skipped (not run)
No individual finding details are sent

Section mode (Fix Suggestions):

Up to 30 individual findings formatted as one-liners
Examples: [HIGH] missing-csp: Add Content-Security-Policy header or [MEDIUM] Missing meta tag: og:description
Severity counts for that section

The AI never sees raw HTML source code, full URLs with credentials, authentication tokens, or personally identifiable information.

Using the AI Score Improvement Plan

Generate a Mega Report and wait for it to complete
In the report header (next to the grade badge), look for the “AI Score Improvement Plan” button with the sparkle icon
Click the button — a loading spinner will appear while the on-device model processes your results
The improvement plan appears in a card below the header, showing:
- Which sections have the most severe findings
- Which section to fix first and why
- Specific severity counts referenced from your scan
- A prioritized action plan to raise your grade
An attribution label reads “Generated on-device by Apple Intelligence”

Using AI Fix Suggestions for Individual Sections

In a completed Mega Report, expand any section (Accessibility, Security, SEO, etc.)
Look for the “AI Suggest Fixes” button with the sparkle icon within the expanded section
Click the button — suggestions are generated on-device in a few seconds
The fix suggestions appear inline below the button, covering:
- What each specific issue is (quoting the exact finding from your scan)
- How to fix it
- The severity impact
The AI only references issues from your actual scan — it does not give generic advice

Supported Test Sections

AI suggestions work with all 19 Mega Report test sections:

Web Testing: Accessibility, Security, Meta Tags, HTML Validation, SEO, Page Size

Link & Content Analysis: Broken Links, Redirect Analysis, Canonical Validation, Hreflang Validation, Structured Data, Image Optimization, Internal Link Structure, Duplicate Content

Browser-based Testing: Console Errors (JS errors, network failures, CORS issues, CSP violations — desktop only)

Code Analysis: Secrets Detection (Gitleaks), Supply Chain (OSV), Semgrep, OpenGrep

Privacy & Data Handling

All processing happens on the device’s Neural Engine via Apple Intelligence
No data is transmitted to Apple, cloud services, or any third party
No internet connection is required for the AI analysis
Prompts contain only severity counts (overall mode) or finding summaries (section mode) — never raw source code, full URLs with credentials, or personally identifiable information
Results are displayed in-app only and are not saved to the database — they are generated fresh each time
“Generated on-device by Apple Intelligence” attribution is shown on all results

Troubleshooting

Button Is Disabled or Grayed Out

Check macOS version — Apple menu > About This Mac — must be macOS 26 or later
Verify Apple Intelligence is enabled — System Settings > Apple Intelligence & Siri
Ensure the report has findings — the AI needs completed scan results to analyze
Hover over the disabled button — the tooltip shows the specific reason it is unavailable

“Apple Intelligence is not available”

This appears when:

Your Mac does not support Apple Intelligence
Apple Intelligence has been disabled in System Settings
The device does not meet the minimum hardware requirements

Check System Settings > Apple Intelligence & Siri to verify status.

AI Returns Generic or Vague Suggestions

The on-device model has a limited context window (~4096 tokens)
If a section has very few findings or only severity counts with no details, the suggestions may be less specific
Sections with more findings produce better, more targeted suggestions

Suggestions Do Not Appear

Ensure the section has completed scanning (status shows completed or failed with findings)
Check the console or logs for error messages
Try generating suggestions for a different section to isolate the issue

Limitations

macOS 26+ only — not available on Windows, iOS, or older macOS versions
30-finding cap — section mode processes up to 30 findings; remaining findings are omitted
~4096 token context window — the on-device model has limited context capacity
Not persistent — AI suggestions are not saved with the report; they are generated fresh each time you click the button
Quality depends on findings — the more detailed the scan findings, the better the AI suggestions
Plain text output — suggestions are plain text, not structured or actionable items

Mega Report — Full documentation for the Mega Report feature
Privacy & External Connections — CodeFrog’s privacy documentation
Security Scanning — Security testing details
Accessibility Testing — Accessibility testing guidance