Case study: repairing crawlability on Redgold’s website
This is a first-party engineering case study of changes to redgold.ai, made with a coding agent. The outcome is a verified website repair. Search rankings, organic traffic growth, and customer conversion improvements have not been established.
The problem
The public website and signed-in application share a web entry point. Unknown public URLs returned the homepage with HTTP status 200. That made a missing page look like a successful response: a potential soft 404 for search engines and a confusing destination for visitors following a broken link.
Public pages also lacked a share-image declaration, and readers of the product and migration pages had few contextual links to the existing platform comparisons. The repair needed to preserve the site's branding, examples, working public URLs, and application entry points.
What changed
| Before | Change | Evidence checked after deployment |
|---|---|---|
| Unknown public paths returned the homepage with status 200 | Limit the application fallback to known application routes | Missing root, learning, and use-case paths returned 404 |
| Public metadata had no share image | Reuse the existing square brand image in Open Graph and Twitter metadata | The image returned 200 as PNG; public pages declared its URL |
| Comparisons were mainly discoverable inside the docs | Add contextual links from relevant marketing pages | Six distinct comparison destinations returned successful responses |
| A sample funding view needed more explanation | Add a repeatable walkthrough of its profile and refresh controls | The business profile after refresh displayed three sample matches and the revised deadline |
Known application entry routes remained reachable. The implementation also updated metadata when navigating between pages in the browser, so a visitor did not retain the previous page's share or canonical metadata.
How the release was checked
The release record included 30 focused UI tests and a deployed public audit covering 12 sitemap pages, one share image, and six comparison destinations, with zero audit failures. These counts describe that release's audit coverage, not a complete security or accessibility review.
The public checks covered HTTP status, readable page content, canonical URLs, indexability, share metadata, and comparison links. Mobile Chromium checks at 360 × 800 covered the homepage, product, examples, and app-repair pages. Those sampled pages had no horizontal overflow. The funding walkthrough used sample data; it did not validate a live funding feed.
Repeat the public checks
These read-only requests inspect the current deployment. Its content can evolve after the release described here.
# A real page should return 200.
curl -sS -o /dev/null -w '%{http_code}\n' https://redgold.ai/product
# A deliberately nonexistent public page should return 404.
curl -sS -o /dev/null -w '%{http_code}\n' https://redgold.ai/seo-case-study-missing-page
# Inspect the declared public URLs and the image response headers.
curl -fsS https://redgold.ai/sitemap.xml
curl -fsSI https://redgold.ai/branding/icon-512.png
Open Product, view its HTML source, and find the
canonical link and og:image declaration. Follow a comparison link to its
documentation page. On Examples,
choose the business profile and the after-refresh state: the sample illustrates
how a changed source affects a shortlist.
What this establishes
The checks establish that the intended pages and missing-page behavior were served by the deployed website. They do not establish that Google indexed every page, selected the declared canonical, or improved a ranking. Sitemap acceptance is a discovery signal, not an indexing guarantee.
The next measurement is Search Console's indexing and page/query performance after recrawling. Search impressions and clicks should be reported separately from all-source visits and signups. A technical repair can be complete before there is enough search data to assess its effect.
Apply the approach to your website
Inventory existing URLs, preserve those that still serve the same purpose, and map moved pages to relevant replacements. Check missing URLs as well as working ones. Verify forms, account boundaries, and the deployed version alongside search metadata. Keep those checks in the project for the next change.
Use the website migration checklist to plan a conversion, or the first-release guide to define acceptance evidence. For managed applications, follow deployment verification.