Google’s /goto Links Change the Cost of SERP Scraping

The click still works. The parser does not

Imagine collecting links from Google for a rank tracker, a research tool, or an internal search index. A few months ago, an unpaid result usually exposed its destination directly in the HTML. Now the same result may point to google.com/goto?url=... instead.

For a person, almost nothing appears broken. You click, Google sends you onward, and the destination page opens. For a scraper, the workflow has changed: reading one results page is no longer enough. Each result may need its own request back to Google before you can learn where it leads.

On August 26, 2026, Google confirmed that it was rolling out new technical measures to protect Search from abuse. By early September, the pattern appeared frequently in logged-out and private browsing sessions, although the exact behavior still varies by browser, account, and search feature. This is best understood as a change to the transport layer of Search, not a change to how pages rank.

What the new link actually contains

A SERP, short for search engine results page, contains the titles, snippets, domains, and links shown after a query. The important detail is that the href attribute for an organic result may now contain Google’s /goto path instead of the final page address.

Google has used redirect wrappers before. The older google.com/url?q=... pattern usually carried a readable, URL-encoded destination. The new url value is different. It is an opaque token, meaning a reference whose contents are not intended to be understood outside the system that created it. It may resemble Base64, a common text encoding, but treating it as Base64 does not reveal a dependable destination.

The useful information appears when Google handles the request:

GET /goto?url=opaque-token HTTP/1.1
Host: www.google.com

HTTP/1.1 302 Found
Location: /the-real-destination

Location is an HTTP response header that tells the client where to go next. The response body is not the interesting part here. Your resolver needs the header, then it should stop.

Google cannot remove every trace of the destination from the results page. It still needs the domain, favicon, attribution, and other details to render the interface. Some pages may also contain embedded data that hints at the real address. Those clues can help with investigation, but they are not a stable contract for production software.

Why one extra hop matters

Google has not published a detailed technical explanation of every part of the rollout. It has described the change as a measure against abuse. The effect on automated collection is clear, though: direct HTML extraction becomes less useful, while bulk URL resolution becomes more visible and more expensive.

A scraper that needs 100 exact destinations may now need roughly 100 additional requests. Those requests add latency, connection management, retries, and failure cases. They also create a recognizable sequence of traffic when one client asks Google to resolve many result links in a short period.

The practical question is: how do you resolve a /goto link without loading the destination page? The answer is to read the redirect response without following it.

Resolve Location, not the destination page

A HEAD request asks for response headers without requesting the normal response body. That makes it a useful first attempt when the only thing you need is Location.

Python’s requests library, a popular HTTP client, can disable automatic redirect following with allow_redirects=False:

from urllib.parse import urljoin
import requests

def resolve_google_goto(goto_href: str) -> str | None:
 options = {
 'allow_redirects': False,
 'timeout': 10,
 }

 with requests.head(goto_href, **options) as response:
 target = response.headers.get('Location')
 if target:
 return urljoin(goto_href, target)

 with requests.get(goto_href, **options) as response:
 target = response.headers.get('Location')
 return urljoin(goto_href, target) if target else None

The second request is a fallback, not an invitation to follow the chain. Some observed responses provide an empty or unhelpful answer to HEAD, while a manual GET exposes the header. A response can also carry Location alongside a status that is not a normal three-hundred redirect, so code should inspect the header before rejecting the response based on its status.

The key line is allow_redirects=False. Without it, the HTTP client may visit the destination page, download content you did not need, and hide the intermediate response that contained the address.

Build for a moving target

The safest production design treats Google’s wrapper as a separate resolution stage. Keep the original result link and the resolved destination in different fields. A useful internal record might contain:

• the raw SERP link
• the resolved destination
• the HTTP status returned by Google
• whether the value came from HEAD or the fallback GET
• the time taken to resolve it

Cache repeated tokens so the same result does not trigger duplicate requests. Use bounded concurrency rather than sending every resolution at once, and retry only temporary failures. The goal is to avoid unnecessary page loads, not to replace one burst of scraping with a larger burst of redirect traffic.

It is also tempting to depend on browser behavior. In some sessions, JavaScript may rewrite result links after the page becomes interactive or after a user moves the pointer over a result. A headless HTML fetch may never trigger that behavior, and Google can rename or remove the underlying data structure without warning. Client-side cleanup is useful for debugging; Location is the more durable signal.

The larger lesson

Google’s /goto rollout does not make search results unreadable. Titles, snippets, visible domains, and ranking positions remain available. What changes is the cost of obtaining the exact destination URL at scale.

That distinction matters for every SERP scraping pipeline. The job is no longer one operation called parse the page. It is two operations: capture the result, then resolve the link through Google without following it. Once that boundary is explicit in your code and data model, the update becomes a protocol change to monitor rather than a mysterious broken parser.

The broader pattern is familiar: search engines are making automated access more deliberate, stateful, and expensive. A resilient scraper does not try to guess what an opaque token means. It records what the server actually says, limits its traffic, and keeps enough diagnostics to adapt when the next redirect format arrives.