Downloading results
Every results page has buttons that download what the search found: URLs, CSV and CSV+snippets. A signed-in account also has Cluster, which saves the results instead of downloading them. Rows come in the same order as on the results page.
What each button gives
| Button | File | Each line |
|---|---|---|
| URLs | query.txt | The address of the page that matched. |
| CSV | query.csv | url;rank |
| CSV+snippets | query-snippets.csv | url;rank;snippets |
| Cluster | - | The domains are saved as a cluster; the message says how many of the results went in. |
The file is named after the query with everything but Latin letters, digits,
- and _ left out: "static.hotjar.com"
gives statichotjarcom.csv.
The format
- UTF-8, no header row.
- Columns are separated by a semicolon
;. A cell that contains a semicolon, a double quote or a line break is put in double quotes, and a quote inside it is doubled. This is standard CSV: a spreadsheet or a CSV library reads it as is, with the semicolon chosen as the separator. - Rank is the site's place in the popularity ranking. It is empty for a site that has no rank.
- All snippets of one page are in one cell, joined with
.... - A snippet that starts with
=,+,-or@gets an apostrophe in front, so a spreadsheet shows it instead of running it as a formula. Numbers such as phone numbers are left as they are.
How many rows
A file holds the results your plan discloses - the rows you can open on the results page - up to the plan's limit: Results / each search or download for URLs and CSV, Download with snippets (rows) for CSV+snippets. The numbers are on the Pricing page. Rows are taken from the top of the results.
When the plan cut the results short, the last line of the file says so:
# Truncated at the plan limit: 1,000 of 22,694 results.
The line starts with # and is written in the language of the site
you downloaded from. It is not a result: skip it when reading the file into a
program. When it is not there, the file holds all the results.
A download is charged like the search itself: one search from the daily allowance, and CSV+snippets also one download with snippets - see quotas and rate limits. When the day's searches are used up, a download carries what the free plan allows, and the last line says the file was cut.
From a script
The buttons build the download address inside the page, and it works for ten
minutes in the browser that opened the page. To download from your own code,
use the API: /v1/search with
format=csv or format=txt. The
old export urls with ?key= keep
their old format exactly: cells are not quoted, there is no last line, and a
site without a rank shows >30M.
Other downloads
- A cluster - the download icon in the list of clusters gives a text file of its domains, one per line.
- Extraction - a CSV separated by commas, not semicolons: the domain, then one column per expression. Several values found on one site share a cell, one per line. When the day's points run out, the last line says where to continue. See Extraction.