User-Agent: * Allow: / Allow: /api/products/ Allow: /api/products/* Allow: /api/lots/ Allow: /api/lots/* Allow: /api/categories/ Allow: /api/categories/* Allow: /api/auth/session Disallow: /api/ Disallow: /profile/ Disallow: /cart/ Disallow: /auth/ Disallow: /admin/ Disallow: /maintenance/ User-Agent: Bytespider User-Agent: CCBot User-Agent: GPTBot User-Agent: ChatGPT-User User-Agent: ClaudeBot User-Agent: Claude-Web User-Agent: anthropic-ai User-Agent: Amazonbot User-Agent: AhrefsBot User-Agent: SemrushBot User-Agent: DotBot User-Agent: MJ12bot User-Agent: PetalBot User-Agent: DataForSeoBot User-Agent: Scrapy Allow: / Allow: /api/products/ Allow: /api/products/* Allow: /api/lots/ Allow: /api/lots/* Allow: /api/categories/ Allow: /api/categories/* Allow: /api/auth/session Disallow: /api/ Disallow: /profile/ Disallow: /cart/ Disallow: /auth/ Disallow: /admin/ Disallow: /maintenance/ User-Agent: Googlebot Allow: / Allow: /api/products/ Allow: /api/products/* Allow: /api/lots/ Allow: /api/lots/* Allow: /api/categories/ Allow: /api/categories/* Allow: /api/auth/session Disallow: /api/ Disallow: /profile/ Disallow: /cart/ Disallow: /auth/ Disallow: /admin/ Disallow: /maintenance/ User-Agent: Bingbot Allow: / Allow: /api/products/ Allow: /api/products/* Allow: /api/lots/ Allow: /api/lots/* Allow: /api/categories/ Allow: /api/categories/* Allow: /api/auth/session Disallow: /api/ Disallow: /profile/ Disallow: /cart/ Disallow: /auth/ Disallow: /admin/ Disallow: /maintenance/ Sitemap: https://www.surplusmarket.com/sitemap.xml Sitemap: https://www.surplusmarket.com/sitemap-pages.xml Sitemap: https://www.surplusmarket.com/sitemap-blogs.xml Sitemap: https://www.surplusmarket.com/sitemap-products.xml