Run a crawl — cr_run • crawlee

Drains the request queue, fetching each request, dispatching it to the matching handler and collecting pushed records, until the queue is empty or the max_requests limit is reached.

Usage

cr_run(crawler)

Arguments

crawler: A configured Crawler.

Value

The crawler, invisibly (its dataset now holds the results).

Examples

if (FALSE) { # \dontrun{
crawler("https://example.com") |>
  cr_on_html(\(ctx) ctx$push_data(list(url = ctx$request$url))) |>
  cr_run() |>
  cr_collect()
} # }