Extract structured data from HTML
Parse a fictional HTML stock table with Cheerio in Node.js, validate its text and links, and copy the JSON into Apps Script to build a restock report.
Extract records from HTML without running the page's scripts, tell parser error recovery apart from missing data, and validate the copied JSON again in Apps Script.
Turn a fictional HTML stock catalog into validated item records, then use those records in Apps Script to calculate stock value and identify items with fewer than five units. The example extracts a table, nested text, a list, link attributes, and embedded JSON. It never visits a website or follows an extracted link.
The parser runs locally in Node.js. Apps Script receives only the resulting JSON, validates it again, and logs a restock report. Apps Script has no npm packages, browser DOM, or Node filesystem, so parsing happens outside it.
Local Apps Script development with clasp covers Node setup. Fetch data from web APIs covers HTTP, which is where a downloaded document would come from. This chapter includes every file for the offline extraction and the Apps Script consumer. It needs no source from earlier examples.
Fetching, parsing, and rendering are different operations
Fetching obtains response bytes. Parsing turns those bytes, after character decoding, into a tree that code can query. A browser does more: it can run JavaScript, load resources, apply CSS, and change the document after the original response arrives.
A selector can find only what exists in the parser's input tree. If a page's initial HTML contains an empty table and JavaScript later fills it, parsing that initial HTML does not produce the rows the browser would add. An HTTP client such as UrlFetchApp does not run page JavaScript either, and waiting longer inside the parser does not make it run.
When an API or structured export exists, prefer it over HTML parsing. Seeing an internal request in browser developer tools does not give you permission to call a private endpoint or reuse a session cookie.
Google's XmlService is a strict XML parser. Ordinary HTML allows omitted closing tags, void elements such as meta, named HTML entities, and browser error-recovery rules that XmlService.parse() rejects. Apps Script's V8 runtime also omits browser DOMParser and Node's URL class.
This project uses Cheerio 1.1.2 in Node.js. Cheerio provides a queryable document tree with a jQuery-like API. By default it uses the spec-compliant HTML parser parse5 (HTML configuration).
Keep the two runtimes and their files separate
| Input or program | Where it belongs | What it does |
|---|---|---|
fixtures/catalog.html | Local input, opened as source text | Supplies fictional markup and data |
tools/extract.mjs and npm dependencies | Node.js on your computer | Parse, select, validate, and write JSON |
output/catalog.json | Generated local output | Carries reviewed data from Node to Apps Script |
HTML_CATALOG_JSON script property | One disposable Apps Script project | Stores the copied JSON text |
apps-script/Code.gs and its manifest | Apps Script | Validate the copied data and log a stock report |
There is no automatic connection between the local directory and the script project. Running the extractor does not update a property, upload code, authorize an account, or deploy anything. Because you copy the data by hand, you can inspect exactly what Apps Script receives.
Assemble the complete input project
Create a new directory outside existing repositories and Apps Script checkouts. In a POSIX-style terminal, such as macOS Terminal or Git Bash, run:
mkdir html-extraction
cd html-extraction
mkdir fixtures tools apps-script Save all eight project files from this chapter into these paths, including the two dotfiles:
html-extraction/
package.json
jsconfig.json
.node-version
.gitignore
fixtures/
catalog.html
tools/
extract.mjs
apps-script/
Code.gs
appsscript.json There is no prewritten output/catalog.json. You generate it by running the extractor against fixtures/catalog.html, the sample input file. There is also no package-lock.json, because npm generates it during installation. If a browser adds .txt to a downloaded filename, remove that suffix when saving the file.
{
"name": "book-html-stock-extraction",
"version": "1.0.0",
"private": true,
"type": "module",
"engines": {
"node": ">=24.21.0 <25"
},
"scripts": {
"extract": "node tools/extract.mjs",
"clean": "node tools/extract.mjs --clean",
"typecheck": "tsc --project jsconfig.json"
},
"dependencies": {
"cheerio": "1.1.2"
},
"devDependencies": {
"@types/node": "24.0.15",
"typescript": "5.9.3"
}
}
24.21.0
Use Node.js 24.21.0 with npm. .node-version records that version for version managers. Check the version your terminal uses before installing.
The project pins Cheerio 1.1.2, Node declarations 24.0.15, and TypeScript 5.9.3. Cheerio ships its own TypeScript declarations, and tsc type-checks the local .mjs extractor.
Review the package file, then run:
node --version
npm --version
npm install --include=dev --ignore-scripts
npm ls --depth=0 Verify that npm ls --depth=0 reports the three declared package versions on Node 24.21.0. --ignore-scripts skips package lifecycle scripts during installation while leaving npm run extract available to run manually.
Review the generated lockfile and keep it with your source. Pinning the three direct dependencies does not pin the packages they depend on, such as the HTML parser and the encoding decoder.
On later clean copies with the reviewed lockfile, use npm ci --include=dev --ignore-scripts. npm documents that this command requires a matching lockfile and replaces an existing node_modules directory. Record Node and npm versions alongside the lockfile when reproducibility matters.
{
"compilerOptions": {
"allowJs": true,
"checkJs": true,
"noEmit": true,
"strict": true,
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"types": ["node"],
"lib": ["ES2022"]
},
"files": ["tools/extract.mjs"]
}
npm run typecheck checks tools/extract.mjs against the Node and Cheerio declarations. Because extract.mjs is standard JavaScript with JSDoc annotations, extraction needs no build or bundling step.
Read the fixture as HTML source
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Workshop stock & prices</title>
<base href="https://untrusted.example/">
<script type="application/json" id="catalog-metadata">
{"currency":"USD","revision":"lesson-1"}
</script>
<script>
throw new Error('Fixture scripts must not execute during extraction.');
</script>
</head>
<body>
<h1>Workshop stock</h1>
<p id="catalog-intro">Fictional <strong>training</strong> stock for the café.</p>
<ul id="stock-notes">
<li>Prices are USD & per unit.
<li>Availability is a snapshot.
</ul>
<table id="stock">
<thead>
<tr><th>Item</th><th>Price</th><th>In stock</th><th>Details</th></tr>
</thead>
<tr data-sku="KIT-001">
<td><span class="item-name">Paper & <em>clips</em></span></td>
<td class="price">$3.50</td>
<td class="quantity">12</td>
<td><a class="details" href="/stock/items/clips.html">View clips</a></td>
</tr>
<tr data-sku="KIT-002">
<td><span class="item-name"><b><i>Café</b> labels</i></span></td>
<td class="price">$2.25</td>
<td class="quantity">8</td>
<td><a class="details" href="items/labels.html?pack=small&lang=en">View labels</a></td>
</tr>
<tr data-sku="KIT-003">
<td><span class="item-name">Labels <refill></span></td>
<td class="price">$1.00</td>
<td class="quantity">0</td>
<td><a class="details" href="/stock/items/refill.html#specs">View refill</a></td>
</tr>
</table>
</body>
</html>
Open this file in a text editor. Do not open untrusted HTML in a browser. The sample file includes an inline <script> that throws if executed. The extractor parses the document tree without running scripts and reads the <script type="application/json"> block with JSON.parse.
The sample HTML tests these parser behaviors:
&, , and<refill>are character references. The output should contain&, normalized spaces, and the literal text<refill>instead of the source spellings.Cafétests non-ASCII text. Themeta charsetdeclares UTF-8 near the start of the file.The list omits
liclosing tags. The table omits an explicittbody, which an HTML parser inserts.The second item misnests
bandi. The parser's HTML error recovery repairs the tree. The extractor does not require well-formed XML.Each row carries a
data-skuattribute. Nested elements inside the item name are part of the name's text and do not create extra records.The
baseelement names a different origin. The extractor ignores it when resolving catalog links and uses its own declared base instead.
The catalog.example and untrusted.example names use the reserved .example suffix. They are reserved for documentation and do not point to working catalog endpoints.
Decode bytes before interpreting characters
The extractor reads the file into a byte buffer of limited size and passes it to Cheerio's loadBuffer. That function detects the character encoding from the bytes the way a browser does, using a byte-order mark or an early character-encoding declaration. The published 1.1.2 implementation accepts the encoding options used here. If the bytes do not identify an encoding, the project uses UTF-8, because you control this local sample file.
If another file has already been decoded incorrectly, an HTML parser cannot reliably recover the original bytes from the damaged string. Preserve raw bytes when character encoding is uncertain. For an approved HTTP response, the charset in the response's Content-Type header also matters. Do not discard it and guess from the filename. The field validator rejects the Unicode replacement character when it appears in extracted text, but that check does not catch every encoding error.
Select structure, then validate its meaning
import { loadBuffer } from 'cheerio';
import { closeSync, fstatSync, lstatSync, mkdirSync, openSync, readSync, renameSync, rmSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { fileURLToPath, URL } from 'node:url';
/** @typedef {{sku: string, name: string, priceCents: number, quantity: number, url: string}} StockItem */
/** @typedef {{schemaVersion: 1, sourceKind: 'local-fixture', revision: string, title: string, introduction: string, currency: 'USD', notes: string[], items: StockItem[]}} StockSnapshot */
const projectRoot = fileURLToPath(new URL('../', import.meta.url));
const inputPath = join(projectRoot, 'fixtures/catalog.html');
const outputDirectory = join(projectRoot, 'output');
const outputPath = join(outputDirectory, 'catalog.json');
const temporaryPath = join(outputDirectory, 'catalog.json.tmp');
const maxInputBytes = 128 * 1024;
const maxOutputBytes = 8000;
const baseUrl = new URL('https://catalog.example/stock/');
const canonicalLink = /^https:\/\/catalog\.example\/stock\/items\/[a-z-]+\.html(?:\?[a-z0-9=&-]+)?$/;
/** @param {unknown} value @returns {value is Record<string, unknown>} */
function isRecord(value) {
return typeof value === 'object' && value !== null && !Array.isArray(value);
}
/** @param {unknown} value @param {string} label @param {number} maximum */
function textField(value, label, maximum) {
if (typeof value !== 'string') {
throw new Error(`${label}: missing text.`);
}
const text = value.replace(/\s+/gu, ' ').trim();
if (!text || text.length > maximum || /[\u0000-\u001f\u007f\ufffd]/u.test(text)) {
throw new Error(`${label}: expected 1 through ${maximum} characters without controls or decoding replacement characters.`);
}
return text;
}
/** @param {unknown} value @param {string} label */
function catalogUrl(value, label) {
if (typeof value !== 'string' || !value || value.length > 512 || /[\u0000-\u0020\u007f]/u.test(value)) {
throw new Error(`${label}: missing or invalid href.`);
}
const url = new URL(value, baseUrl);
if (url.origin !== baseUrl.origin || url.username || url.password) {
throw new Error(`${label}: link must use the approved HTTPS origin without credentials.`);
}
url.hash = '';
if (!canonicalLink.test(url.href) || url.href.length > 512) {
throw new Error(`${label}: unsupported catalog path or query.`);
}
return url.href;
}
/** @param {string} text @param {string} label */
function priceInCents(text, label) {
const match = /^\$(0|[1-9][0-9]{0,2})\.([0-9]{2})$/.exec(text);
if (!match) {
throw new Error(`${label}: expected a USD price such as $3.50.`);
}
const cents = Number(match[1]) * 100 + Number(match[2]);
if (cents > 10000) {
throw new Error(`${label}: price exceeds $100.00.`);
}
return cents;
}
/** @returns {Buffer} */
function readBoundedInput() {
const entry = lstatSync(inputPath);
if (!entry.isFile() || entry.isSymbolicLink()) {
throw new Error('fixtures/catalog.html must be an ordinary file, not a symlink.');
}
const descriptor = openSync(inputPath, 'r');
try {
const stat = fstatSync(descriptor);
if (!stat.isFile() || stat.size > maxInputBytes) {
throw new Error('HTML input exceeds the 128 KiB file limit.');
}
const buffer = Buffer.alloc(maxInputBytes + 1);
let length = 0;
while (length < buffer.length) {
const count = readSync(descriptor, buffer, length, buffer.length - length, null);
if (count === 0) break;
length += count;
}
if (length === 0 || length > maxInputBytes) {
throw new Error('HTML input must contain 1 through 131072 bytes.');
}
return buffer.subarray(0, length);
} finally {
closeSync(descriptor);
}
}
/** @param {Buffer} input @returns {StockSnapshot} */
function extractSnapshot(input) {
const $ = loadBuffer(input, {
scriptingEnabled: false,
encoding: { defaultEncoding: 'utf-8' },
});
const metadataElements = $('script#catalog-metadata[type="application/json"]');
if (metadataElements.length !== 1) {
throw new Error('Expected exactly one JSON catalog-metadata element.');
}
/** @type {unknown} */
const metadata = JSON.parse(metadataElements.text());
if (!isRecord(metadata) || metadata.currency !== 'USD') {
throw new Error('Catalog metadata must declare currency USD.');
}
const revision = textField(metadata.revision, 'Catalog revision', 32);
if (!/^[a-z0-9-]+$/.test(revision)) {
throw new Error('Catalog revision must use lowercase letters, digits, and hyphens.');
}
// Exclude non-content nodes from text extraction, not to execute or sanitize HTML.
$('script, style, template, noscript').remove();
if ($('title').length !== 1 || $('#catalog-intro').length !== 1 || $('ul#stock-notes').length !== 1) {
throw new Error('Expected one title, catalog-intro paragraph, and stock-notes list.');
}
const title = textField($('title').text(), 'Title', 120);
const introduction = textField($('#catalog-intro').text(), 'Introduction', 240);
const noteElements = $('ul#stock-notes').children('li');
if (noteElements.length < 1 || noteElements.length > 5) {
throw new Error('Expected 1 through 5 stock notes.');
}
const notes = noteElements.toArray().map((element, index) => textField($(element).text(), `Note ${index + 1}`, 160));
const table = $('table#stock');
if (table.length !== 1 || table.find('table, [rowspan], [colspan]').length !== 0 || table.children('tfoot').length !== 0) {
throw new Error('Expected one stock table without nested tables, merged cells, or a footer.');
}
const headerRows = table.children('thead').children('tr');
const expectedHeaders = ['Item', 'Price', 'In stock', 'Details'];
const headers = headerRows.children('th').toArray().map((element) => textField($(element).text(), 'Header', 40));
if (table.children('thead').length !== 1 || headerRows.length !== 1 || headerRows.children().length !== 4 ||
headers.length !== 4 || headers.some((header, index) => header !== expectedHeaders[index])) {
throw new Error('Stock headers must be Item, Price, In stock, Details.');
}
const bodies = table.children('tbody');
const rows = bodies.children('tr');
if (bodies.length !== 1 || rows.length < 1 || rows.length > 20) {
throw new Error('Expected one stock body with 1 through 20 item rows.');
}
/** @type {StockItem[]} */
const items = [];
/** @type {Set<string>} */
const seen = new Set();
rows.each((index, element) => {
const row = $(element);
const label = `Item row ${index + 1}`;
const sku = textField(row.attr('data-sku'), `${label} SKU`, 7);
if (!/^KIT-[0-9]{3}$/.test(sku) || seen.has(sku)) {
throw new Error(`${label}: SKU must be unique and have the form KIT-001.`);
}
seen.add(sku);
const cells = row.children('td');
if (cells.length !== 4 || row.children().length !== 4 || !cells.eq(1).hasClass('price') || !cells.eq(2).hasClass('quantity')) {
throw new Error(`${label}: expected four unmerged cells with price and quantity in their declared columns.`);
}
const nameElement = cells.eq(0).find('.item-name');
const linkElement = cells.eq(3).find('a.details');
if (nameElement.length !== 1 || linkElement.length !== 1) {
throw new Error(`${label}: expected one item-name and one details link.`);
}
const name = textField(nameElement.text(), `${label} name`, 120);
const priceCents = priceInCents(textField(cells.eq(1).text(), `${label} price`, 12), label);
const quantityText = textField(cells.eq(2).text(), `${label} quantity`, 3);
if (!/^(0|[1-9][0-9]{0,2})$/.test(quantityText) || Number(quantityText) > 100) {
throw new Error(`${label}: quantity must be an integer from 0 through 100.`);
}
const url = catalogUrl(linkElement.attr('href'), `${label} details`);
items.push({ sku, name, priceCents, quantity: Number(quantityText), url });
});
items.sort((a, b) => a.sku < b.sku ? -1 : a.sku > b.sku ? 1 : 0);
return { schemaVersion: 1, sourceKind: 'local-fixture', revision, title, introduction, currency: 'USD', notes, items };
}
function prepareOutputDirectory() {
const entry = lstatSync(outputDirectory, { throwIfNoEntry: false });
if (entry && (!entry.isDirectory() || entry.isSymbolicLink())) {
throw new Error('output must be an ordinary directory, not a file or symlink.');
}
if (!entry) mkdirSync(outputDirectory);
}
function cleanGeneratedFiles() {
const paths = [outputPath, temporaryPath];
for (const path of paths) {
const entry = lstatSync(path, { throwIfNoEntry: false });
if (entry && (!entry.isFile() || entry.isSymbolicLink())) {
throw new Error('Refusing cleanup: a generated output path is not an ordinary file.');
}
}
for (const path of paths) rmSync(path, { force: true });
}
const args = process.argv.slice(2);
if (args.length > 1 || (args.length === 1 && args[0] !== '--clean')) {
throw new Error('Use npm run extract or npm run clean; no input URL or output path is accepted.');
}
prepareOutputDirectory();
cleanGeneratedFiles();
if (args[0] === '--clean') {
console.log('Removed only output/catalog.json and output/catalog.json.tmp.');
} else {
try {
const snapshot = extractSnapshot(readBoundedInput());
const json = JSON.stringify(snapshot, null, 2) + '\n';
if (Buffer.byteLength(json, 'utf8') > maxOutputBytes) {
throw new Error('JSON output exceeds the 8000-byte transfer limit.');
}
writeFileSync(temporaryPath, json, { encoding: 'utf8', flag: 'wx' });
renameSync(temporaryPath, outputPath);
console.log(`Extracted ${snapshot.items.length} items -> output/catalog.json (no network requests).`);
} catch (error) {
cleanGeneratedFiles();
throw error;
}
}
A CSS selector identifies elements by structure. table#stock selects the stock table, .item-name selects the element with that class inside a cell, and a.details selects the details anchor. children('td') examines only direct cells, not every nested descendant. The selectors are fixed in the source, and the program never builds one from text in the page.
Text and attributes are read differently. Cheerio's .text() collects descendant text, so Paper, a nonbreaking space, &, and nested clips become one string. .attr('href') reads an anchor's attribute value, after HTML parsing has decoded character references. It does not fetch the link.
The extractor reads the application/json script block before removing script, style, template, and noscript nodes from the tree used for ordinary text extraction. This keeps code and fallback markup out of the text fields. Cheerio does not execute page scripts or load page resources, even if you leave those nodes in place. It also does not apply CSS. Text hidden by a stylesheet is still text in the parsed tree.
Parsing can succeed even when the document is no longer a usable catalog. The program therefore requires:
One JSON metadata script block with currency
USDand a revision label within the length limit.One title, introduction, notes list, and stock table.
The exact four headers, one table body, and 1 through 20 item rows. An empty table is an error. The extractor does not read it as zero stock.
No nested table, merged cells, or table footer in the selected table.
A unique
KIT-SKU with three digits, one name, one details link, and the price and quantity in their declared columns.Normalized text within the length limits, a complete USD price string, an integer quantity from 0 through 100, and a link in the allowed catalog format.
The regular expressions validate already-extracted strings such as a SKU, price, quantity, or canonical URL. They do not find HTML tags or rebuild a document tree. The extractor fails on a missing field instead of filling in an empty string or zero.
Prices become integer cents: $3.50 becomes 350. The stock-value calculation then adds whole numbers instead of binary floating-point dollar values. For this sample file, the extractor rejects $3.50 each, negative prices, an unsupported currency, and values over $100.00. If a real source uses another price format, change and review the price rule. Do not use parseFloat, which accepts any string that starts with a valid-looking number.
Resolve links without trusting the page's base
The extractor uses the URL constructor from node:url. It resolves raw href values against https://catalog.example/stock/, then checks the resulting origin and rejects embedded credentials. A base URL does not restrict anything by itself, because an absolute input URL overrides it. Check the origin after parsing to reject URLs on another host.
The final string must use an allowed item path and a limited query format. The program drops fragment identifiers, because each catalog record points to an item page as a whole. For the second item, expect:
https://catalog.example/stock/items/labels.html?pack=small&lang=en The & in the HTML attribute becomes &, and the relative path is resolved against the extractor's own base. A javascript: link, an HTTP link, a different origin, an embedded username or password, or an unsupported path makes extraction fail. A page-controlled base element does not change these decisions.
These checks examine only the link strings, and the extractor never fetches the links. They would not protect a future downloader from server-side request forgery (SSRF). A real downloader would also need to check the destination of each connection and each redirect, because checking a string does not control DNS resolution or a later redirect.
Produce and inspect the actual JSON
From the local project directory, run:
npm run extract Expect the success message to report three extracted items and output/catalog.json. Open that file in your editor. Do not create a JSON file by copying an expected-result table from this chapter.
The output contains schema version 1, source kind local-fixture, revision lesson-1, currency USD, the title and introduction, two notes, and three item objects sorted by SKU. Expected item values are:
| SKU | Name | priceCents | quantity |
|---|---|---|---|
KIT-001 | Paper & clips | 350 | 12 |
KIT-002 | Café labels | 225 | 8 |
KIT-003 | Labels <refill> | 100 | 0 |
Verify that the title is Workshop stock & prices, the introduction is Fictional training stock for the café., and the notes contain decoded text instead of tags. Inspect all three canonical link strings. The inline <script> that throws should not have caused an execution error during extraction.
The output is pretty-printed JSON with a trailing newline, keys in a fixed order, items sorted by SKU, and no timestamp. Running the same pinned project against unchanged bytes should reproduce the same output. It is a snapshot of the sample file, and the sourceKind field (local-fixture) records that it came from a local file.
Limit input size and output files
The extractor accepts only its fixed fixtures/catalog.html path. It reads at most 131,073 bytes (one byte more than the 128 KiB input cap) so it can detect and reject a larger file before parsing. The fixed selectors and the caps on record counts and text lengths also restrict the data it accepts.
Before extracting, it removes only output/catalog.json and its temporary file, output/catalog.json.tmp. A successful run validates all records, writes the temporary file, and renames it to catalog.json. When the extractor handles an error, it removes partial output, so a failed run does not leave an older output file in place. A forcibly interrupted process can leave a temporary file, so clean and rerun before transferring data.
The output directory and the two output paths must be an ordinary folder and ordinary files. They must not be symlinks, and a folder must not sit where a file is expected. Do not run extract and clean at the same time, and never keep hand-maintained data in either generated file. The program does not delete arbitrary contents of output.
Copy the JSON into Apps Script
The Apps Script half uses no parser library. Only these two files belong in the remote project:
/** @typedef {{sku: string, name: string, priceCents: number, quantity: number, url: string}} CatalogItem */
/** @typedef {{schemaVersion: 1, sourceKind: 'local-fixture', revision: string, title: string, introduction: string, currency: 'USD', notes: string[], items: CatalogItem[]}} CatalogSnapshot */
/** @param {unknown} value @returns {value is Record<string, unknown>} */
function isCatalogRecord_(value) {
return typeof value === 'object' && value !== null && !Array.isArray(value);
}
/** @param {unknown} value @param {string} label @param {number} maximum */
function catalogText_(value, label, maximum) {
if (typeof value !== 'string' || !value || value.length > maximum ||
value !== value.replace(/\s+/gu, ' ').trim() || /[\u0000-\u001f\u007f\ufffd]/u.test(value)) {
throw new Error(label + ': missing, oversized, or non-normalized text.');
}
return value;
}
/** @param {unknown} value @param {string} label @param {number} maximum */
function catalogInteger_(value, label, maximum) {
if (typeof value !== 'number' || !Number.isInteger(value) || value < 0 || value > maximum) {
throw new Error(label + ': expected an integer from 0 through ' + maximum + '.');
}
return value;
}
/** @param {unknown} value @returns {CatalogSnapshot} */
function validateCatalogSnapshot_(value) {
if (!isCatalogRecord_(value) || value.schemaVersion !== 1 || value.sourceKind !== 'local-fixture' || value.currency !== 'USD') {
throw new Error('Expected a version-1 local-fixture catalog in USD.');
}
const revision = catalogText_(value.revision, 'Revision', 32);
if (!/^[a-z0-9-]+$/.test(revision)) {
throw new Error('Revision must use lowercase letters, digits, and hyphens.');
}
const title = catalogText_(value.title, 'Title', 120);
const introduction = catalogText_(value.introduction, 'Introduction', 240);
if (!Array.isArray(value.notes) || value.notes.length < 1 || value.notes.length > 5) {
throw new Error('Expected 1 through 5 catalog notes.');
}
/** @type {unknown[]} */
const noteValues = value.notes;
const notes = noteValues.map(function (note, index) {
return catalogText_(note, 'Note ' + (index + 1), 160);
});
if (!Array.isArray(value.items) || value.items.length < 1 || value.items.length > 20) {
throw new Error('Expected 1 through 20 catalog items.');
}
/** @type {unknown[]} */
const itemValues = value.items;
/** @type {CatalogItem[]} */
const items = [];
/** @type {Set<string>} */
const seen = new Set();
for (let index = 0; index < itemValues.length; index += 1) {
const item = itemValues[index];
const label = 'Item ' + (index + 1);
if (!isCatalogRecord_(item)) {
throw new Error(label + ': expected an object.');
}
const sku = catalogText_(item.sku, label + ' SKU', 7);
if (!/^KIT-[0-9]{3}$/.test(sku) || seen.has(sku)) {
throw new Error(label + ': SKU must be unique and have the form KIT-001.');
}
seen.add(sku);
const name = catalogText_(item.name, label + ' name', 120);
const priceCents = catalogInteger_(item.priceCents, label + ' priceCents', 10000);
const quantity = catalogInteger_(item.quantity, label + ' quantity', 100);
const url = catalogText_(item.url, label + ' URL', 512);
if (!/^https:\/\/catalog\.example\/stock\/items\/[a-z-]+\.html(?:\?[a-z0-9=&-]+)?$/.test(url)) {
throw new Error(label + ': URL is outside the allowed canonical catalog link format.');
}
items.push({ sku: sku, name: name, priceCents: priceCents, quantity: quantity, url: url });
}
items.sort(function (a, b) { return a.sku < b.sku ? -1 : a.sku > b.sku ? 1 : 0; });
return {
schemaVersion: 1,
sourceKind: 'local-fixture',
revision: revision,
title: title,
introduction: introduction,
currency: 'USD',
notes: notes,
items: items
};
}
function logCatalogRestockPlan() {
const json = PropertiesService.getScriptProperties().getProperty('HTML_CATALOG_JSON');
if (!json) {
throw new Error('Generate output/catalog.json locally, then copy its complete contents into the HTML_CATALOG_JSON script property.');
}
if (json.length > 8000 || Utilities.newBlob(json).getBytes().length > 8000) {
throw new Error('HTML_CATALOG_JSON exceeds the 8000-byte transfer limit.');
}
/** @type {unknown} */
let parsed;
try {
parsed = JSON.parse(json);
} catch (error) {
throw new Error(
'HTML_CATALOG_JSON is not valid JSON. Copy the complete contents of output/catalog.json into the script property.'
);
}
const snapshot = validateCatalogSnapshot_(parsed);
let units = 0;
let stockValueCents = 0;
for (const item of snapshot.items) {
units += item.quantity;
stockValueCents += item.priceCents * item.quantity;
}
const restock = snapshot.items.filter(function (item) {
return item.quantity < 5;
}).map(function (item) {
return { sku: item.sku, name: item.name, quantity: item.quantity, url: item.url };
});
Logger.log(JSON.stringify({
revision: snapshot.revision,
title: snapshot.title,
currency: snapshot.currency,
itemCount: snapshot.items.length,
units: units,
stockValueCents: stockValueCents,
restock: restock
}));
}
{
"timeZone": "Etc/UTC",
"dependencies": {},
"exceptionLogging": "STACKDRIVER",
"runtimeVersion": "V8",
"oauthScopes": []
}
The manifest selects the V8 runtime and declares an empty list of OAuth scopes. The consumer reads a script property, counts its UTF-8 bytes in memory, validates JSON, and logs a report. It does not request access to Sheets, Drive, email, or an external website. Installing npm packages does not grant any Google permission. Do not copy the Node extractor into Code.gs. Its imports are not available in Apps Script.
To run the Apps Script consumer in a disposable project:
Open
script.google.comwith the approved account and create a standalone project namedHTML stock extraction - test. Do not reuse a project with existing data or automation.Replace its default
Code.gscontents with the completeapps-script/Code.gssource.In Project Settings, enable Show "appsscript.json" manifest file in editor. Replace the disposable project's manifest with the supplied manifest and save. No deployment, trigger, Apps Script library, or advanced service is needed.
Run the local extractor successfully and inspect
output/catalog.jsonas above.In the test project's Project Settings, under Script Properties, click Add script property. Set Property to
HTML_CATALOG_JSONand Value to the complete generated JSON text. Click Save script properties.In the Apps Script editor, select
logCatalogRestockPlan, click Run, and inspect Execution log. If the editor asks for authorization, check the project name and account before you approve. If it asks for broad service permissions you did not expect, stop and check which project you are running.
Script properties store strings and are shared within their project. Other scripts cannot read them. Here the property holds a small amount of copied data. Do not paste executable expressions or confidential scraped pages into it. Anyone who can edit the project can change this property. For larger records, design a different way to store or transfer the data. Do not try to raise the property limit or split a page of unknown size across many keys.
JavaScript string.length counts UTF-16 code units rather than UTF-8 bytes, so multi-byte characters such as é take more storage than .length reports. The extractor limits its UTF-8 JSON output to 8000 bytes, and the consumer checks the same byte count with Utilities.newBlob(text).getBytes().length to stay below the 9 KB per-property value limit. Quotas can change, and the smaller project limit does not override Google's limit.
Validate again after the transfer
JSON.parse checks JSON syntax, not whether a value is a stock catalog. The consumer first treats its result as unknown. Before it builds typed records, its validation checks the schema version, source kind, currency, text length limits, and array shapes. It also requires a unique SKU format, integer cents, integer quantities, and the exact canonical link format.
The JSDoc types describe the validated record shape for tooling. The comments check nothing at run time. Only the validation code checks the copied data. A manually edited property containing "priceCents": "350" is still valid JSON but is not a valid number field.
The consumer's regular expression verifies the canonical https://catalog.example/stock/items/... URL format produced by the Node step without fetching those URLs.
Only after the entire input passes validation does Apps Script calculate totals and select items whose quantity is below five. Expect the baseline report to contain:
itemCountof3andunitsof20.stockValueCentsof6000, representing12 * 350 + 8 * 225 + 0 * 100, or USD60.00.One restock record,
KIT-003, with the text nameLabels <refill>, quantity0, and its canonical link.
Running again with the same saved property should produce the same report. The script does not write cells, change a property, order stock, or send a message.
Predict, change, fail, and restore
Change one quantity and copy it again
In fixtures/catalog.html, change only the KIT-002 quantity from 8 to 2. Predict the new totals before running anything: 14 units and 4650 cents, with both KIT-002 and KIT-003 below the restock threshold.
Run npm run extract and inspect the changed JSON. Before you replace the property, run logCatalogRestockPlan again. Expect the old 20-unit report, because the local file and the saved property are separate copies. Replace HTML_CATALOG_JSON with the complete new JSON, save, and run again. Expect the predicted new totals and two restock records.
Restore the sample file's quantity to 8, extract again, inspect, replace the property with the restored output, and run to recover the baseline. Editing the local file alone does not restore the remote property.
Distinguish HTML recovery from missing business data
First repair the misnested formatting in the second item so the opening and closing b and i elements nest correctly. Predict no change to the extracted name Café labels. Extract and compare the output. The business text should remain the same even though the source markup changed. Restore the original sample file afterward if you want later results to match the baseline.
Next, remove only the href attribute from the KIT-002 details anchor. Keep the anchor and its class. The document is still parseable HTML, but the required link is missing. Run extraction and expect a nonzero exit with:
Item row 2 details: missing or invalid href. There should be no output/catalog.json after this handled failure. Do not copy JSON from an old editor tab as if extraction had produced a new file. Restore the original href, including its & spelling in HTML, and run extraction again. The baseline JSON should return.
A missing table, changed header, duplicate SKU, two instead of a quantity, or merged cell also breaks the rules the extractor checks. Do not make these errors disappear by skipping rows or substituting zero. Inspect the source structure and decide whether the source or the extractor's schema needs to change.
Reject a bad value after parsing JSON
In the disposable script property's JSON, temporarily change the first item's numeric priceCents value 350 to the string "350". Save and run the consumer. Expect an integer-validation error for the first item's priceCents and no successful stock report from that execution.
Restore the whole property from a fresh successful baseline extraction instead of guessing which fields you changed. Run again and expect the baseline totals. Anyone who can edit the project can change the property after transfer, so the Apps Script consumer must validate it even though the local extractor checked the original output.
Treat parsed content as untrusted data
Cheerio's security guidance distinguishes HTML parsing from sanitization. A parsed tree can still contain <script> elements, event-handler attributes, and javascript: URLs if serialized back into HTML.
Even plain text extracted with .text() can contain <, >, quotes, or a leading = that triggers a spreadsheet formula. This consumer logs only values within its length limits. If you later render extracted strings in HTML or write them to Sheets, apply contextual escaping or text-only cell formatting. Never pass extracted values to eval, dynamic imports, scriptlet evaluation, or raw HTML insertion.
Do not accept page-supplied selectors, filenames, destinations, or executable instructions. The known input path, fixed selectors, field limits, and output paths limit what this exercise can read and write. For untrusted production input, run the parser in a restricted process with explicit limits on time, memory, and other resources.
Obtain permission before adding a real fetch
This offline example is complete without a downloader. Before adapting it to a real site, obtain permission for the source and intended use. Review the terms of use and your copyright and data-retention obligations. Avoid collecting personal or confidential information you do not need.
Respect the site's published robots.txt rules and access controls. RFC 9309 says robots rules are not access authorization. A path that is not disallowed is not a license to bypass login, a paywall, a CAPTCHA, a private API policy, or another restriction. Do not rotate identities or impersonate another client to get around a refusal.
A permitted download step needs its own limits and checks before it hands bytes to the parser:
Select approved HTTPS origins and paths, validate certificates, and do not accept arbitrary user-provided URLs. Check each redirect and each resolved destination so requests cannot reach local, private, or metadata-service addresses.
Check status and media type before parsing. An HTML login or error page returned with an HTTP success status is still not the catalog. The structural checks should fail instead of producing empty results.
Limit request duration, redirect count, downloaded and decompressed bytes, and total requests. Keep the response's character-encoding information. Reject a large
Content-Length, and still enforce a byte limit while reading the response.Respect publisher rate limits and
Retry-Afterguidance, cache or retain an approved snapshot when appropriate, and stop on authentication or access failures. A parsing failure is not a reason to repeatedly fetch the same page.
Do not add pagination, recursive link following, scheduling, or browser execution to this small extractor as incidental improvements. Each one changes how much the extractor fetches, what it is allowed to access, and how it can fail, so each needs its own review. If an approved API already returns the needed structured data, validate that response directly instead of scraping a rendered page.
Clean only this example's outputs
node_modules/
/output/
*.log
.DS_Store
# No live targets, environment files, or credentials belong in this example.
.clasp.json
.clasp.*.json
.clasprc.json
.clasprc.*.json
.env
.env.*
.npmrc
credentials*.json
client_secret*.json
service-account*.json
*.pem
*.key
Keep the sample HTML, scripts, manifest, configuration, and lockfile while you work through the chapter. To remove the generated JSON output once no extraction is running:
npm run clean The command removes output/catalog.json and output/catalog.json.tmp while leaving the source files and node_modules intact. When finished with the disposable Apps Script project, remove HTML_CATALOG_JSON in Project Settings > Script Properties or move the standalone test project to Trash.
Next, read Keep one Sheets chart up to date.
Project files
Complete source files for this chapter's examples.
Extract a fictional HTML stock catalog locally and validate it in Apps Script
package.json
Download file{
"name": "book-html-stock-extraction",
"version": "1.0.0",
"private": true,
"type": "module",
"engines": {
"node": ">=24.21.0 <25"
},
"scripts": {
"extract": "node tools/extract.mjs",
"clean": "node tools/extract.mjs --clean",
"typecheck": "tsc --project jsconfig.json"
},
"dependencies": {
"cheerio": "1.1.2"
},
"devDependencies": {
"@types/node": "24.0.15",
"typescript": "5.9.3"
}
}
jsconfig.json
Download file{
"compilerOptions": {
"allowJs": true,
"checkJs": true,
"noEmit": true,
"strict": true,
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"types": ["node"],
"lib": ["ES2022"]
},
"files": ["tools/extract.mjs"]
}
.node-version
Download file24.21.0
.gitignore
Download filenode_modules/
/output/
*.log
.DS_Store
# No live targets, environment files, or credentials belong in this example.
.clasp.json
.clasp.*.json
.clasprc.json
.clasprc.*.json
.env
.env.*
.npmrc
credentials*.json
client_secret*.json
service-account*.json
*.pem
*.key
fixtures/catalog.html
Download file<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Workshop stock & prices</title>
<base href="https://untrusted.example/">
<script type="application/json" id="catalog-metadata">
{"currency":"USD","revision":"lesson-1"}
</script>
<script>
throw new Error('Fixture scripts must not execute during extraction.');
</script>
</head>
<body>
<h1>Workshop stock</h1>
<p id="catalog-intro">Fictional <strong>training</strong> stock for the café.</p>
<ul id="stock-notes">
<li>Prices are USD & per unit.
<li>Availability is a snapshot.
</ul>
<table id="stock">
<thead>
<tr><th>Item</th><th>Price</th><th>In stock</th><th>Details</th></tr>
</thead>
<tr data-sku="KIT-001">
<td><span class="item-name">Paper & <em>clips</em></span></td>
<td class="price">$3.50</td>
<td class="quantity">12</td>
<td><a class="details" href="/stock/items/clips.html">View clips</a></td>
</tr>
<tr data-sku="KIT-002">
<td><span class="item-name"><b><i>Café</b> labels</i></span></td>
<td class="price">$2.25</td>
<td class="quantity">8</td>
<td><a class="details" href="items/labels.html?pack=small&lang=en">View labels</a></td>
</tr>
<tr data-sku="KIT-003">
<td><span class="item-name">Labels <refill></span></td>
<td class="price">$1.00</td>
<td class="quantity">0</td>
<td><a class="details" href="/stock/items/refill.html#specs">View refill</a></td>
</tr>
</table>
</body>
</html>
tools/extract.mjs
Download fileimport { loadBuffer } from 'cheerio';
import { closeSync, fstatSync, lstatSync, mkdirSync, openSync, readSync, renameSync, rmSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { fileURLToPath, URL } from 'node:url';
/** @typedef {{sku: string, name: string, priceCents: number, quantity: number, url: string}} StockItem */
/** @typedef {{schemaVersion: 1, sourceKind: 'local-fixture', revision: string, title: string, introduction: string, currency: 'USD', notes: string[], items: StockItem[]}} StockSnapshot */
const projectRoot = fileURLToPath(new URL('../', import.meta.url));
const inputPath = join(projectRoot, 'fixtures/catalog.html');
const outputDirectory = join(projectRoot, 'output');
const outputPath = join(outputDirectory, 'catalog.json');
const temporaryPath = join(outputDirectory, 'catalog.json.tmp');
const maxInputBytes = 128 * 1024;
const maxOutputBytes = 8000;
const baseUrl = new URL('https://catalog.example/stock/');
const canonicalLink = /^https:\/\/catalog\.example\/stock\/items\/[a-z-]+\.html(?:\?[a-z0-9=&-]+)?$/;
/** @param {unknown} value @returns {value is Record<string, unknown>} */
function isRecord(value) {
return typeof value === 'object' && value !== null && !Array.isArray(value);
}
/** @param {unknown} value @param {string} label @param {number} maximum */
function textField(value, label, maximum) {
if (typeof value !== 'string') {
throw new Error(`${label}: missing text.`);
}
const text = value.replace(/\s+/gu, ' ').trim();
if (!text || text.length > maximum || /[\u0000-\u001f\u007f\ufffd]/u.test(text)) {
throw new Error(`${label}: expected 1 through ${maximum} characters without controls or decoding replacement characters.`);
}
return text;
}
/** @param {unknown} value @param {string} label */
function catalogUrl(value, label) {
if (typeof value !== 'string' || !value || value.length > 512 || /[\u0000-\u0020\u007f]/u.test(value)) {
throw new Error(`${label}: missing or invalid href.`);
}
const url = new URL(value, baseUrl);
if (url.origin !== baseUrl.origin || url.username || url.password) {
throw new Error(`${label}: link must use the approved HTTPS origin without credentials.`);
}
url.hash = '';
if (!canonicalLink.test(url.href) || url.href.length > 512) {
throw new Error(`${label}: unsupported catalog path or query.`);
}
return url.href;
}
/** @param {string} text @param {string} label */
function priceInCents(text, label) {
const match = /^\$(0|[1-9][0-9]{0,2})\.([0-9]{2})$/.exec(text);
if (!match) {
throw new Error(`${label}: expected a USD price such as $3.50.`);
}
const cents = Number(match[1]) * 100 + Number(match[2]);
if (cents > 10000) {
throw new Error(`${label}: price exceeds $100.00.`);
}
return cents;
}
/** @returns {Buffer} */
function readBoundedInput() {
const entry = lstatSync(inputPath);
if (!entry.isFile() || entry.isSymbolicLink()) {
throw new Error('fixtures/catalog.html must be an ordinary file, not a symlink.');
}
const descriptor = openSync(inputPath, 'r');
try {
const stat = fstatSync(descriptor);
if (!stat.isFile() || stat.size > maxInputBytes) {
throw new Error('HTML input exceeds the 128 KiB file limit.');
}
const buffer = Buffer.alloc(maxInputBytes + 1);
let length = 0;
while (length < buffer.length) {
const count = readSync(descriptor, buffer, length, buffer.length - length, null);
if (count === 0) break;
length += count;
}
if (length === 0 || length > maxInputBytes) {
throw new Error('HTML input must contain 1 through 131072 bytes.');
}
return buffer.subarray(0, length);
} finally {
closeSync(descriptor);
}
}
/** @param {Buffer} input @returns {StockSnapshot} */
function extractSnapshot(input) {
const $ = loadBuffer(input, {
scriptingEnabled: false,
encoding: { defaultEncoding: 'utf-8' },
});
const metadataElements = $('script#catalog-metadata[type="application/json"]');
if (metadataElements.length !== 1) {
throw new Error('Expected exactly one JSON catalog-metadata element.');
}
/** @type {unknown} */
const metadata = JSON.parse(metadataElements.text());
if (!isRecord(metadata) || metadata.currency !== 'USD') {
throw new Error('Catalog metadata must declare currency USD.');
}
const revision = textField(metadata.revision, 'Catalog revision', 32);
if (!/^[a-z0-9-]+$/.test(revision)) {
throw new Error('Catalog revision must use lowercase letters, digits, and hyphens.');
}
// Exclude non-content nodes from text extraction, not to execute or sanitize HTML.
$('script, style, template, noscript').remove();
if ($('title').length !== 1 || $('#catalog-intro').length !== 1 || $('ul#stock-notes').length !== 1) {
throw new Error('Expected one title, catalog-intro paragraph, and stock-notes list.');
}
const title = textField($('title').text(), 'Title', 120);
const introduction = textField($('#catalog-intro').text(), 'Introduction', 240);
const noteElements = $('ul#stock-notes').children('li');
if (noteElements.length < 1 || noteElements.length > 5) {
throw new Error('Expected 1 through 5 stock notes.');
}
const notes = noteElements.toArray().map((element, index) => textField($(element).text(), `Note ${index + 1}`, 160));
const table = $('table#stock');
if (table.length !== 1 || table.find('table, [rowspan], [colspan]').length !== 0 || table.children('tfoot').length !== 0) {
throw new Error('Expected one stock table without nested tables, merged cells, or a footer.');
}
const headerRows = table.children('thead').children('tr');
const expectedHeaders = ['Item', 'Price', 'In stock', 'Details'];
const headers = headerRows.children('th').toArray().map((element) => textField($(element).text(), 'Header', 40));
if (table.children('thead').length !== 1 || headerRows.length !== 1 || headerRows.children().length !== 4 ||
headers.length !== 4 || headers.some((header, index) => header !== expectedHeaders[index])) {
throw new Error('Stock headers must be Item, Price, In stock, Details.');
}
const bodies = table.children('tbody');
const rows = bodies.children('tr');
if (bodies.length !== 1 || rows.length < 1 || rows.length > 20) {
throw new Error('Expected one stock body with 1 through 20 item rows.');
}
/** @type {StockItem[]} */
const items = [];
/** @type {Set<string>} */
const seen = new Set();
rows.each((index, element) => {
const row = $(element);
const label = `Item row ${index + 1}`;
const sku = textField(row.attr('data-sku'), `${label} SKU`, 7);
if (!/^KIT-[0-9]{3}$/.test(sku) || seen.has(sku)) {
throw new Error(`${label}: SKU must be unique and have the form KIT-001.`);
}
seen.add(sku);
const cells = row.children('td');
if (cells.length !== 4 || row.children().length !== 4 || !cells.eq(1).hasClass('price') || !cells.eq(2).hasClass('quantity')) {
throw new Error(`${label}: expected four unmerged cells with price and quantity in their declared columns.`);
}
const nameElement = cells.eq(0).find('.item-name');
const linkElement = cells.eq(3).find('a.details');
if (nameElement.length !== 1 || linkElement.length !== 1) {
throw new Error(`${label}: expected one item-name and one details link.`);
}
const name = textField(nameElement.text(), `${label} name`, 120);
const priceCents = priceInCents(textField(cells.eq(1).text(), `${label} price`, 12), label);
const quantityText = textField(cells.eq(2).text(), `${label} quantity`, 3);
if (!/^(0|[1-9][0-9]{0,2})$/.test(quantityText) || Number(quantityText) > 100) {
throw new Error(`${label}: quantity must be an integer from 0 through 100.`);
}
const url = catalogUrl(linkElement.attr('href'), `${label} details`);
items.push({ sku, name, priceCents, quantity: Number(quantityText), url });
});
items.sort((a, b) => a.sku < b.sku ? -1 : a.sku > b.sku ? 1 : 0);
return { schemaVersion: 1, sourceKind: 'local-fixture', revision, title, introduction, currency: 'USD', notes, items };
}
function prepareOutputDirectory() {
const entry = lstatSync(outputDirectory, { throwIfNoEntry: false });
if (entry && (!entry.isDirectory() || entry.isSymbolicLink())) {
throw new Error('output must be an ordinary directory, not a file or symlink.');
}
if (!entry) mkdirSync(outputDirectory);
}
function cleanGeneratedFiles() {
const paths = [outputPath, temporaryPath];
for (const path of paths) {
const entry = lstatSync(path, { throwIfNoEntry: false });
if (entry && (!entry.isFile() || entry.isSymbolicLink())) {
throw new Error('Refusing cleanup: a generated output path is not an ordinary file.');
}
}
for (const path of paths) rmSync(path, { force: true });
}
const args = process.argv.slice(2);
if (args.length > 1 || (args.length === 1 && args[0] !== '--clean')) {
throw new Error('Use npm run extract or npm run clean; no input URL or output path is accepted.');
}
prepareOutputDirectory();
cleanGeneratedFiles();
if (args[0] === '--clean') {
console.log('Removed only output/catalog.json and output/catalog.json.tmp.');
} else {
try {
const snapshot = extractSnapshot(readBoundedInput());
const json = JSON.stringify(snapshot, null, 2) + '\n';
if (Buffer.byteLength(json, 'utf8') > maxOutputBytes) {
throw new Error('JSON output exceeds the 8000-byte transfer limit.');
}
writeFileSync(temporaryPath, json, { encoding: 'utf8', flag: 'wx' });
renameSync(temporaryPath, outputPath);
console.log(`Extracted ${snapshot.items.length} items -> output/catalog.json (no network requests).`);
} catch (error) {
cleanGeneratedFiles();
throw error;
}
}
apps-script/Code.gs
Download file/** @typedef {{sku: string, name: string, priceCents: number, quantity: number, url: string}} CatalogItem */
/** @typedef {{schemaVersion: 1, sourceKind: 'local-fixture', revision: string, title: string, introduction: string, currency: 'USD', notes: string[], items: CatalogItem[]}} CatalogSnapshot */
/** @param {unknown} value @returns {value is Record<string, unknown>} */
function isCatalogRecord_(value) {
return typeof value === 'object' && value !== null && !Array.isArray(value);
}
/** @param {unknown} value @param {string} label @param {number} maximum */
function catalogText_(value, label, maximum) {
if (typeof value !== 'string' || !value || value.length > maximum ||
value !== value.replace(/\s+/gu, ' ').trim() || /[\u0000-\u001f\u007f\ufffd]/u.test(value)) {
throw new Error(label + ': missing, oversized, or non-normalized text.');
}
return value;
}
/** @param {unknown} value @param {string} label @param {number} maximum */
function catalogInteger_(value, label, maximum) {
if (typeof value !== 'number' || !Number.isInteger(value) || value < 0 || value > maximum) {
throw new Error(label + ': expected an integer from 0 through ' + maximum + '.');
}
return value;
}
/** @param {unknown} value @returns {CatalogSnapshot} */
function validateCatalogSnapshot_(value) {
if (!isCatalogRecord_(value) || value.schemaVersion !== 1 || value.sourceKind !== 'local-fixture' || value.currency !== 'USD') {
throw new Error('Expected a version-1 local-fixture catalog in USD.');
}
const revision = catalogText_(value.revision, 'Revision', 32);
if (!/^[a-z0-9-]+$/.test(revision)) {
throw new Error('Revision must use lowercase letters, digits, and hyphens.');
}
const title = catalogText_(value.title, 'Title', 120);
const introduction = catalogText_(value.introduction, 'Introduction', 240);
if (!Array.isArray(value.notes) || value.notes.length < 1 || value.notes.length > 5) {
throw new Error('Expected 1 through 5 catalog notes.');
}
/** @type {unknown[]} */
const noteValues = value.notes;
const notes = noteValues.map(function (note, index) {
return catalogText_(note, 'Note ' + (index + 1), 160);
});
if (!Array.isArray(value.items) || value.items.length < 1 || value.items.length > 20) {
throw new Error('Expected 1 through 20 catalog items.');
}
/** @type {unknown[]} */
const itemValues = value.items;
/** @type {CatalogItem[]} */
const items = [];
/** @type {Set<string>} */
const seen = new Set();
for (let index = 0; index < itemValues.length; index += 1) {
const item = itemValues[index];
const label = 'Item ' + (index + 1);
if (!isCatalogRecord_(item)) {
throw new Error(label + ': expected an object.');
}
const sku = catalogText_(item.sku, label + ' SKU', 7);
if (!/^KIT-[0-9]{3}$/.test(sku) || seen.has(sku)) {
throw new Error(label + ': SKU must be unique and have the form KIT-001.');
}
seen.add(sku);
const name = catalogText_(item.name, label + ' name', 120);
const priceCents = catalogInteger_(item.priceCents, label + ' priceCents', 10000);
const quantity = catalogInteger_(item.quantity, label + ' quantity', 100);
const url = catalogText_(item.url, label + ' URL', 512);
if (!/^https:\/\/catalog\.example\/stock\/items\/[a-z-]+\.html(?:\?[a-z0-9=&-]+)?$/.test(url)) {
throw new Error(label + ': URL is outside the allowed canonical catalog link format.');
}
items.push({ sku: sku, name: name, priceCents: priceCents, quantity: quantity, url: url });
}
items.sort(function (a, b) { return a.sku < b.sku ? -1 : a.sku > b.sku ? 1 : 0; });
return {
schemaVersion: 1,
sourceKind: 'local-fixture',
revision: revision,
title: title,
introduction: introduction,
currency: 'USD',
notes: notes,
items: items
};
}
function logCatalogRestockPlan() {
const json = PropertiesService.getScriptProperties().getProperty('HTML_CATALOG_JSON');
if (!json) {
throw new Error('Generate output/catalog.json locally, then copy its complete contents into the HTML_CATALOG_JSON script property.');
}
if (json.length > 8000 || Utilities.newBlob(json).getBytes().length > 8000) {
throw new Error('HTML_CATALOG_JSON exceeds the 8000-byte transfer limit.');
}
/** @type {unknown} */
let parsed;
try {
parsed = JSON.parse(json);
} catch (error) {
throw new Error(
'HTML_CATALOG_JSON is not valid JSON. Copy the complete contents of output/catalog.json into the script property.'
);
}
const snapshot = validateCatalogSnapshot_(parsed);
let units = 0;
let stockValueCents = 0;
for (const item of snapshot.items) {
units += item.quantity;
stockValueCents += item.priceCents * item.quantity;
}
const restock = snapshot.items.filter(function (item) {
return item.quantity < 5;
}).map(function (item) {
return { sku: item.sku, name: item.name, quantity: item.quantity, url: item.url };
});
Logger.log(JSON.stringify({
revision: snapshot.revision,
title: snapshot.title,
currency: snapshot.currency,
itemCount: snapshot.items.length,
units: units,
stockValueCents: stockValueCents,
restock: restock
}));
}
apps-script/appsscript.json
Download file{
"timeZone": "Etc/UTC",
"dependencies": {},
"exceptionLogging": "STACKDRIVER",
"runtimeVersion": "V8",
"oauthScopes": []
}