Skip to main content

​
Overview

Scrape services provide powerful tools for extracting content from websites, discovering sitemaps, and running specialized scraping actors.

​
Available Services

Website Scraping

Scrape websites and convert to markdown

Sitemap Discovery

Find all pages on a domain ranked by relevance

​
Common Use Cases

​
Basic Website Scraping

const result = await services.scrape.website({
   url: "https://example.com",
   params: { limit: 1 },
});

console.log(result.markdown);

​
Find Relevant Pages

const pages = await services.scrape.sitemap({
   domain: "docs.stripe.com",
   keywords: ["payment", "API", "integration"],
});

// Pages are ranked by relevance to keywords
pages.forEach((page) => {
   console.log(`${page.title}: ${page.url}`);
});

​
Extract Social URLs

const result = await services.scrape.website({
   url: "https://company.com",
});

console.log("Social Media:");
console.log("LinkedIn:", result.socialUrls.linkedinCompany);
console.log("Twitter:", result.socialUrls.twitterUser);
console.log("Email:", result.socialUrls.emailGeneral);