Production setup is mostly about predictable requests and clear failure handling.

Checklist

• Store API keys in secrets, not source code.
• Set request timeouts.
• Add retry logic only for retryable failures.
• Log request IDs and status codes.
• Keep source links in the user-facing result.
• Cache stable scrape results when it makes product sense.

Retry shape

Use short retries with backoff for temporary network failures. Do not retry authentication errors without changing credentials.

User messaging

If a request fails, tell the user whether the issue is authentication, rate limit, upstream retrieval, or temporary service availability.