Skip to main content
GET
/
integrations
/
web_crawler
/
index
JavaScript
import Hyperspell from 'hyperspell';

const client = new Hyperspell({
  apiKey: process.env['HYPERSPELL_API_KEY'], // This is the default and can be omitted
});

const response = await client.integrations.webCrawler.index({ url: 'url' });

console.log(response.resource_id);
{
  "source": "reddit",
  "resource_id": "<string>",
  "status": "pending"
}

Authorizations

​
Authorization
string
header
required

API Key or JWT User Token. If using an API Key, set the X-As-User header to act as a specific user. A JWT User Token is always scoped to a specific user.

Query Parameters

​
url
string
required

The base URL of the website to crawl

​
max_depth
integer
default:2

Maximum depth of links to follow during crawling

Required range: 0 <= x <= 5
​
limit
integer
default:20

Maximum number of pages to crawl in total

Required range: 1 <= x <= 100

Response

Successful Response

​
source
enum<string>
required
Available options:
reddit,
notion,
slack,
google_calendar,
google_mail,
box,
dropbox,
google_drive,
github,
vault,
web_crawler,
trace,
microsoft_teams
​
resource_id
string
required
​
status
enum<string>
required
Available options:
pending,
processing,
completed,
failed,
pending_review,
skipped