Fetch data from web APIs
Use UrlFetchApp.fetch() in Apps Script to import a JSON weather forecast into a Sheets range and read a limited number of GitHub API pages.
Build and inspect real HTTP requests, validate their responses before using them, and stop a paginated read at a set limit while reporting that the result is partial.
Fetch data from web APIs
Import seven days of public weather data into a reserved Sheets range, then read a fixed number of pages from GitHub's public REST API and report when the result is a partial sample. These are two separate projects. The weather script writes to a spreadsheet, and the pagination script only logs public repository metadata. You need runtime and permissions and Read and write Sheets in batches first.
An HTTP server can reject your request, apply a rate limit, or return different data on a later run. Check every HTTP response before you write its values to a sheet.
Describe the request before sending it
An HTTP request is a message sent to a server to read or change a resource. Apps Script sends HTTP requests through UrlFetchApp. The browser fetch API is not available. Use the built-in or advanced Google services for Google APIs, and UrlFetchApp for other web APIs.
Each request has five main parts:
| Part | Example | Meaning |
|---|---|---|
| Method | GET | Reads a resource without modifying remote state. |
| URL | https://api.open-meteo.com/v1/forecast | The HTTPS scheme, host, and endpoint path. |
| Query | latitude=52.52&longitude=13.41 | Key-value parameters after ?, separated by &. |
| Headers | Accept: application/json | Request metadata. Accept requests JSON from the server. |
| Body | None in these GET requests | Payload sent with write methods such as POST. |
An endpoint is one URL path and HTTP method that an API offers. HTTPS encrypts the request in transit. The server still receives, and may log, the full URL, query parameters, and headers.
When you build a query string from variables, encode each parameter value on its own with encodeURIComponent. Do not encode the whole URL. Encoding each value keeps a character such as & inside a value from adding an extra parameter.
Preview a request without contacting its URL
UrlFetchApp.getRequest() returns the request object that UrlFetchApp.fetch() would send, without opening a network connection.
function previewRequest() {
const baseUrl = 'https://example.com/api/requests';
const query = 'status=open&limit=10';
const request = UrlFetchApp.getRequest(baseUrl + '?' + query, {
headers: { Accept: 'application/json' },
method: 'get',
});
Logger.log(JSON.stringify(request, null, 2));
}
{
"timeZone": "Etc/UTC",
"runtimeVersion": "V8",
"oauthScopes": [
"https://www.googleapis.com/auth/script.external_request"
]
}
{
"baseUrl": "https://example.com/api/requests",
"query": {
"status": "open",
"limit": 10
},
"headers": {
"Accept": "application/json"
}
}
Create a new standalone project at
script.google.comand replaceCode.gswith the preview file.In Project Settings, enable Show "appsscript.json" manifest file in editor and replace
appsscript.jsonwith the included manifest.sample-request.jsonshows the expected output. Do not add it to Apps Script.Select
previewRequest, save, and click Run. The manifest declaresscript.external_requestbecausegetRequestrequires that scope even though it makes no network call.In the Execution log, find the formatted JSON object. It shows the
GETURLhttps://example.com/api/requests?status=open&limit=10and theAcceptheader.Change
status=opentostatus=resolvedinquery, save, and run again. The logged URL changes. Then restorestatus=open.
Separate Google consent from remote authentication
Both live projects declare https://www.googleapis.com/auth/script.external_request in appsscript.json (scope guide). That Google OAuth scope lets Apps Script send outbound HTTP requests. It does not identify you to the remote API. The weather project also declares https://www.googleapis.com/auth/spreadsheets.currentonly to write to its bound sheet.
A remote API that needs authentication expects its own credentials, such as an API key or an Authorization header. Those credentials are separate from Google OAuth consent. Neither Open-Meteo's free forecast endpoint nor GitHub's public tags endpoint needs credentials. Never send a Google OAuth token to a third-party API.
Outbound requests come from Google's URL Fetch IP pool, not from your browser's IP address (UrlFetchApp reference).
Check status, content, and data as separate steps
HTTPResponse provides getResponseCode() for the numeric HTTP status, getAllHeaders() for the response headers, and getContentText('UTF-8') for the response body text.
An HTTP error status such as 404 or 503 is still a response, and you can inspect it. A DNS failure, a TLS error, a missing Google scope, or an exhausted Apps Script quota throws an exception before any response arrives.
By default, UrlFetchApp.fetch throws on an HTTP 4xx or 5xx status. Both projects pass muteHttpExceptions: true, read getResponseCode() themselves, and require 200. Both also pass followRedirects: false, so an unexpected 3xx redirect stops the run.
Before they parse the body, the helpers check the response's Content-Type header, ignoring case and parameters such as charset=utf-8. Proxies and gateways often return an HTML error page even when the request asked for Accept: application/json.
JSON.parse turns a JSON string into a JavaScript value. When the body is not JSON, JSON.parse throws a syntax error, which try and catch catch (see JSON.parse errors). Parsing does not check the data's structure or field types. Valid JSON can still lack an expected array or contain text where a number should be. Each project checks those conditions before using the data.
Use Open-Meteo within its free-service terms
The forecast project requests daily high and low temperatures for Berlin (latitude=52.52, longitude=13.41) from Open-Meteo's forecast API. It passes timezone=auto for local calendar dates and temperature_unit=celsius.
Open-Meteo's free tier currently allows fewer than 10,000 calls per day, 5,000 per hour, and 600 per minute. Its CC BY 4.0 licence requires attribution, a licence link, and a note that says whether the data was changed. The script writes those three attribution rows below the forecast table, in A9:B11.
The GET request sends the coordinates in the URL. Open-Meteo's privacy policy notes that API logs may record the requested coordinates, so keep the public Berlin coordinates for this exercise.
Reserve the forecast destination before the first run
Create a disposable spreadsheet in which the script owns Forecast!A1:C12:
const FORECAST_DAYS = 7;
const FORECAST_LATITUDE = 52.52;
const FORECAST_LONGITUDE = 13.41;
const FORECAST_OWNER_MARKER = 'Owned by weather-import';
/**
* A property-access view for validation, not a validated provider response.
* Every data field remains unknown until the checks in forecastRows succeed.
* @typedef {{daily?: {time?: unknown, temperature_2m_max?: unknown, temperature_2m_min?: unknown}|null, daily_units?: {temperature_2m_max?: unknown, temperature_2m_min?: unknown}|null}} ForecastCandidate
*/
function importForecastToSheet() {
if (!Number.isInteger(FORECAST_DAYS) || FORECAST_DAYS < 1 || FORECAST_DAYS > 7) {
throw new Error('Set FORECAST_DAYS to a whole number from 1 to 7.');
}
if (!Number.isFinite(FORECAST_LATITUDE) || Math.abs(FORECAST_LATITUDE) > 90 ||
!Number.isFinite(FORECAST_LONGITUDE) || Math.abs(FORECAST_LONGITUDE) > 180) {
throw new Error('Use numeric latitude from -90 to 90 and longitude from -180 to 180.');
}
const spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
if (!spreadsheet) {
throw new Error('Open this project through Extensions > Apps Script in the practice spreadsheet.');
}
const sheet = spreadsheet.getSheetByName('Forecast');
if (!sheet) {
throw new Error('Create a blank tab named Forecast before running this function.');
}
const target = sheet.getRange('A1:C12');
checkForecastDestination(target, sheet);
const query = [
'latitude=' + encodeURIComponent(FORECAST_LATITUDE),
'longitude=' + encodeURIComponent(FORECAST_LONGITUDE),
'daily=temperature_2m_max,temperature_2m_min',
'temperature_unit=celsius',
'timezone=auto',
'forecast_days=' + FORECAST_DAYS
].join('&');
const response = UrlFetchApp.fetch('https://api.open-meteo.com/v1/forecast?' + query, {
method: 'get',
headers: { Accept: 'application/json' },
muteHttpExceptions: true,
followRedirects: false
});
const statusCode = response.getResponseCode();
if (statusCode !== 200) {
throw new Error('The forecast service answered with status code ' + statusCode + '. No cells were written.');
}
const contentType = forecastHeader(response.getAllHeaders(), 'content-type');
if (contentType.split(';')[0].trim().toLowerCase() !== 'application/json') {
throw new Error('The forecast response was not application/json. No cells were written.');
}
const text = response.getContentText('UTF-8');
if (text.length > 50000) {
throw new Error('The forecast response exceeded the 50000-character parsing limit.');
}
/** @type {unknown} */
let forecast;
try {
forecast = JSON.parse(text);
} catch (error) {
throw new Error('The forecast response was not valid JSON. No cells were written.');
}
const rows = forecastRows(forecast, FORECAST_DAYS);
// Check again after the request in case the destination changed while it was pending.
checkForecastDestination(target, sheet);
target.setValues(rows);
Logger.log('Wrote ' + FORECAST_DAYS + ' days of forecast.');
}
/**
* @param {SpreadsheetApp.Range} target
* @param {SpreadsheetApp.Sheet} sheet
* @returns {void}
*/
function checkForecastDestination(target, sheet) {
if (target.isPartOfMerge()) {
throw new Error('Forecast!A1:C12 must not overlap merged cells.');
}
if (target.isBlank()) return;
const markerCell = sheet.getRange('A12');
if (markerCell.getValue() !== FORECAST_OWNER_MARKER || markerCell.getFormula() !== '') {
throw new Error('Forecast!A1:C12 is not blank and has no valid ownership marker. Use a new practice sheet; do not clear existing data to bypass this check.');
}
}
/**
* @param {unknown} rawForecast
* @param {number} expectedDays
* @returns {Array<Array<string|number>>}
*/
function forecastRows(rawForecast, expectedDays) {
// This view permits property probes; the original runtime checks still validate the value.
const forecast = /** @type {ForecastCandidate|null|undefined} */ (rawForecast);
if (!forecast || typeof forecast !== 'object' || Array.isArray(forecast) ||
!forecast.daily || !forecast.daily_units) {
throw new Error('The forecast response is missing daily data or units.');
}
const days = forecast.daily.time;
const highs = forecast.daily.temperature_2m_max;
const lows = forecast.daily.temperature_2m_min;
if (!Array.isArray(days) || !Array.isArray(highs) || !Array.isArray(lows) ||
days.length !== expectedDays || highs.length !== expectedDays || lows.length !== expectedDays) {
throw new Error('The forecast arrays must each contain exactly ' + expectedDays + ' days.');
}
if (forecast.daily_units.temperature_2m_max !== '°C' ||
forecast.daily_units.temperature_2m_min !== '°C') {
throw new Error('The forecast temperature units must be Celsius.');
}
/** @type {Array<Array<string|number>>} */
const rows = [['Date', 'High', 'Low']];
let index = 0;
/** @type {number|null} */
let previousDay = null;
while (index < expectedDays) {
/** @type {unknown} */
const day = days[index];
/** @type {unknown} */
const high = highs[index];
/** @type {unknown} */
const low = lows[index];
if (typeof day !== 'string' || !/^\d{4}-\d{2}-\d{2}$/.test(day)) {
throw new Error('Forecast day ' + (index + 1) + ' is not a YYYY-MM-DD date.');
}
const timestamp = Date.parse(day + 'T00:00:00Z');
if (!Number.isFinite(timestamp) || new Date(timestamp).toISOString().slice(0, 10) !== day ||
(previousDay !== null && timestamp - previousDay !== 86400000)) {
throw new Error('Forecast dates must be valid consecutive calendar days.');
}
if (!Number.isFinite(high) || !Number.isFinite(low) ||
/** @type {number} */ (high) < /** @type {number} */ (low)) {
throw new Error('Forecast day ' + (index + 1) + ' needs numeric high and low temperatures with high >= low.');
}
rows.push([day, /** @type {number} */ (high), /** @type {number} */ (low)]);
previousDay = timestamp;
index = index + 1;
}
// The fixed output area has room for seven days, even when fewer are requested.
while (rows.length < 8) {
rows.push(['', '', '']);
}
rows.push(['Weather data by Open-Meteo', 'https://open-meteo.com/', '']);
rows.push(['Licence: CC BY 4.0', 'https://creativecommons.org/licenses/by/4.0/', '']);
rows.push(['Selected daily highs/lows; Celsius; local dates', '', '']);
rows.push([FORECAST_OWNER_MARKER, '', '']);
return rows;
}
/**
* @param {Record<string, string|string[]>} headers
* @param {string} wantedName
* @returns {string}
*/
function forecastHeader(headers, wantedName) {
for (const name of Object.keys(headers)) {
if (name.toLowerCase() === wantedName) {
const value = headers[name];
if (Array.isArray(value)) return value.join(', ');
return String(value);
}
}
return '';
}
{
"timeZone": "Etc/UTC",
"dependencies": {},
"exceptionLogging": "STACKDRIVER",
"runtimeVersion": "V8",
"oauthScopes": [
"https://www.googleapis.com/auth/script.external_request",
"https://www.googleapis.com/auth/spreadsheets.currentonly"
]
}
Create a new spreadsheet named
Forecast practice, rename its tabForecast, and leaveA1:C12blank and unmerged.Open Extensions > Apps Script and replace
Code.gswith the complete weather-import file.In Project Settings, enable Show "appsscript.json" manifest file in editor, replace
appsscript.jsonwith the included manifest, and save both files.Keep
FORECAST_DAYS = 7,FORECAST_LATITUDE = 52.52, andFORECAST_LONGITUDE = 13.41.Select
importForecastToSheet, click Run, and approve the current-spreadsheet and external-request permissions.Inspect
Forecast!A1:C12and Execution log when the run completes.
The script checks the destination before sending the HTTP request and again before calling setValues() (Range reference). It requires:
A tab named
Forecast.No merged cells overlapping
A1:C12.A1:C12to be completely blank, or to have the script's ownership marker inA12:Owned by weather-import.
Once A12 holds that marker, later runs overwrite A1:C12.
Check the written forecast range
With a valid seven-day response, Forecast!A1:C12 contains:
| Cells | Expected contents |
|---|---|
A1:C1 | Date, High, Low |
A2:C8 | Seven consecutive local-date labels with numeric daily high and low temperatures in Celsius |
A9:B9 | Open-Meteo credit and source URL |
A10:B10 | CC BY 4.0 label and licence URL |
A11 | Selection, units, and local-date description |
A12 | Owned by weather-import |
The execution log displays:
Wrote 7 days of forecast. The endpoint returns live model data, so the temperatures and dates change from day to day. The returned coordinates may also be those of the nearest weather-model grid point.
Before calling setValues(), the script checks the 200 status, the JSON content type, and a 50,000-character body limit. It also checks that the array lengths equal FORECAST_DAYS, the units are Celsius, the date strings are valid consecutive YYYY-MM-DD values, and the high and low temperatures are finite numbers with high >= low.
The output array is always 12 rows by 3 columns. When fewer than seven days are requested, the script fills the unused rows through row 8 with '', so the same setValues() call clears stale rows from an earlier run. Date labels must match YYYY-MM-DD and temperatures must be finite numbers, so a remote string that starts with = can never reach the sheet as a formula.
Read the validation helpers
forecastRows checks each date label in three steps:
The regular expression
/^\d{4}-\d{2}-\d{2}$/requires a four-digit year, a two-digit month, and a two-digit day, separated by hyphens.Date.parse(day + 'T00:00:00Z')parses the label at UTC midnight.new Date(timestamp).toISOString().slice(0, 10) === daythen confirms that the date did not roll over from an invalid calendar day such as2026-02-30.Each timestamp must be exactly
86400000milliseconds (one day) after the previous one, so the dates are consecutive.
In forecastHeader, Object.keys(headers) returns the header names, and a for...of loop compares each name to the wanted one, ignoring case.
Request fewer days and remove stale output
Change
FORECAST_DAYSfrom7to3.Predict the layout before you run: row 1 holds the header,
A2:C4holds three forecast days,A5:C8is blank, andA9:C12keeps the attribution and marker rows.Save and run
importForecastToSheet. The log showsWrote 3 days of forecast., andA5:C8is blank.Restore
FORECAST_DAYS = 7, save, and run again to restore the seven-day table.
To test the input check, set FORECAST_DAYS to 8 and run importForecastToSheet. The function throws Set FORECAST_DAYS to a whole number from 1 to 7. before it sends a request or touches the sheet. Restore 7 afterward.
Recover without erasing the previous forecast
| Failure | What to inspect and what remains |
|---|---|
No active spreadsheet, no Forecast tab, or a destination that is merged or lacks the ownership marker | Open the bound project from the Forecast practice sheet. These checks run before the HTTP request. |
fetch throws without a response | Read the error message. The cause can be missing OAuth consent, an exhausted quota, or a network failure. No cells were written. |
HTTP 400 | Open-Meteo returns 400 for invalid query parameters. Fix the parameters before you run again. The previous sheet values are untouched. |
HTTP 429 or 5xx | Stop and wait before you retry. The forecast script does not retry on its own. |
| Unexpected content type, invalid JSON, or invalid data fields | The checks reject the response before setValues(), and the previous forecast stays in place. |
| The run stopped during the final write | Inspect Forecast!A1:C12 before you run again. |
Read a small sample from a paginated API
A page is one slice of a larger collection. GitHub's list-repository-tags endpoint returns tag records in pages, controlled by the per_page and page query parameters. The second project fetches up to two pages of five public tags each from microsoft/TypeScript. Each request sends Accept: application/vnd.github+json, a User-Agent header, and X-GitHub-Api-Version: 2026-03-10 (GitHub API versions).
GitHub allows unauthenticated reads of public repositories at up to 60 requests per hour per source IP address. Your requests leave from Google's outbound IP pool, so that budget is shared with other Apps Script callers.
const TAG_MAX_PAGES = 2;
const TAG_MAX_ITEMS = 10;
const TAG_PAGE_SIZE = 5;
const TAG_ENDPOINT = 'https://api.github.com/repos/microsoft/TypeScript/tags';
/** @typedef {{name: string, commit: {sha: string}}} RepositoryTag */
/**
* A property-access view only; name and SHA are unknown until readTagPage validates them.
* @typedef {{name?: unknown, commit?: {sha?: unknown}|null}|null|undefined} TagCandidate
*/
function listPublicRepositoryTags() {
if (!Number.isInteger(TAG_MAX_PAGES) || TAG_MAX_PAGES < 1 || TAG_MAX_PAGES > 3 ||
!Number.isInteger(TAG_MAX_ITEMS) || TAG_MAX_ITEMS < 1 || TAG_MAX_ITEMS > 15 ||
TAG_PAGE_SIZE !== 5) {
throw new Error('Use 1-3 pages, 1-15 items, and a page size of 5.');
}
const progress = { httpAttempts: 0 };
const tags = [];
let pagesRead = 0;
/** @type {string|null} */
let url = TAG_ENDPOINT + '?per_page=' + TAG_PAGE_SIZE + '&page=1';
let complete = false;
let stopReason = '';
/** @type {unknown} */
let failure = null;
try {
while (url !== null) {
if (pagesRead >= TAG_MAX_PAGES) {
stopReason = 'page limit reached; more pages available';
break;
}
if (tags.length >= TAG_MAX_ITEMS) {
stopReason = 'item limit reached; more data available';
break;
}
checkTagPageUrl(url, pagesRead + 1);
const response = fetchTagPage(url, progress);
const headers = response.getAllHeaders();
const page = readTagPage(response, headers);
const nextUrl = nextTagPageUrl(headers);
if (nextUrl !== null) checkTagPageUrl(nextUrl, pagesRead + 2);
const room = TAG_MAX_ITEMS - tags.length;
for (const tag of page) {
if (tags.length >= TAG_MAX_ITEMS) break;
tags.push({ name: tag.name, sha: tag.commit.sha });
}
pagesRead = pagesRead + 1;
if (page.length > room) {
stopReason = 'item limit reached within a validated page';
break;
}
url = nextUrl;
if (url === null) {
complete = true;
stopReason = 'no next link reported';
} else if (pagesRead < TAG_MAX_PAGES && tags.length < TAG_MAX_ITEMS &&
tagHeader(headers, 'x-ratelimit-remaining') === '0') {
throw new Error('GitHub reports no requests remaining. ' + tagWaitDetails(headers));
}
}
} catch (error) {
failure = error;
stopReason = String(error);
}
let resultLabel = 'partial';
if (complete) resultLabel = 'complete';
Logger.log('Repository: microsoft/TypeScript');
Logger.log('Pages accepted: ' + pagesRead + '; HTTP attempts: ' + progress.httpAttempts);
Logger.log('Tags collected: ' + tags.length);
Logger.log('Result: ' + resultLabel + '; ' + stopReason);
Logger.log('Tag names: ' + JSON.stringify(tags.map((tag) => tag.name)));
if (failure !== null) throw failure;
}
/**
* @param {string} url
* @param {{httpAttempts: number}} progress
* @returns {UrlFetchApp.HTTPResponse}
*/
function fetchTagPage(url, progress) {
let attempt = 0;
while (attempt < 3) {
attempt = attempt + 1;
progress.httpAttempts = progress.httpAttempts + 1;
// Exceptions without an HTTP response are not automatically retried.
const response = UrlFetchApp.fetch(url, {
method: 'get',
headers: {
Accept: 'application/vnd.github+json',
'X-GitHub-Api-Version': '2026-03-10',
'User-Agent': 'apps-script-book-http-pagination'
},
muteHttpExceptions: true,
followRedirects: false
});
const status = response.getResponseCode();
const headers = response.getAllHeaders();
if (status === 200) return response;
if (status === 403 || status === 429 ||
tagHeader(headers, 'retry-after') !== '' ||
tagHeader(headers, 'x-ratelimit-remaining') === '0') {
throw new Error('GitHub HTTP ' + status + '; no automatic retry. ' + tagWaitDetails(headers));
}
const transient = status === 500 || status === 502 || status === 503 || status === 504;
if (!transient || attempt === 3) {
throw new Error('GitHub HTTP ' + status + ' after ' + attempt + ' attempt(s) for this page. Requests stopped.');
}
// Only these read-only GET failures are retried: wait 1 second, then 2 seconds.
Utilities.sleep(attempt * 1000);
}
throw new Error('The page attempt limit was reached.');
}
/**
* @param {UrlFetchApp.HTTPResponse} response
* @param {Record<string, string|string[]>} headers
* @returns {RepositoryTag[]}
*/
function readTagPage(response, headers) {
const contentType = tagHeader(headers, 'content-type').split(';')[0].trim().toLowerCase();
if (contentType !== 'application/json' && contentType !== 'application/vnd.github+json') {
throw new Error('GitHub returned an unexpected content type.');
}
const text = response.getContentText('UTF-8');
if (text.length > 100000) {
throw new Error('The GitHub response exceeded the 100000-character parsing limit.');
}
/** @type {unknown} */
let page;
try {
page = JSON.parse(text);
} catch (error) {
throw new Error('GitHub returned invalid JSON.');
}
if (!Array.isArray(page) || page.length > TAG_PAGE_SIZE) {
throw new Error('Expected an array containing at most five GitHub tags.');
}
for (const tag of /** @type {TagCandidate[]} */ (page)) {
if (!tag || typeof tag !== 'object' || Array.isArray(tag) ||
typeof tag.name !== 'string' || tag.name.length === 0 || tag.name.length > 256 ||
!tag.commit || typeof tag.commit.sha !== 'string' ||
!/^[0-9a-f]{40}$/.test(tag.commit.sha)) {
throw new Error('A GitHub tag is missing its expected name or commit SHA.');
}
}
return /** @type {RepositoryTag[]} */ (page);
}
/**
* @param {Record<string, string|string[]>} headers
* @returns {string|null}
*/
function nextTagPageUrl(headers) {
const link = tagHeader(headers, 'link');
if (link === '') return null;
if (link.length > 8000) throw new Error('GitHub pagination headers exceeded this example\'s limit.');
/** @type {string|null} */
let next = null;
for (const entry of link.split(',')) {
const match = entry.trim().match(/^<([^<>]+)>;\s*rel="(next|prev|first|last)"$/);
if (!match) throw new Error('GitHub returned an unsupported pagination link format.');
if (match[2] === 'next') {
if (next !== null) throw new Error('GitHub returned more than one next-page link.');
next = match[1];
}
}
return next;
}
/**
* @param {string} url
* @param {number} expectedPage
* @returns {void}
*/
function checkTagPageUrl(url, expectedPage) {
// GitHub may use this repository's numeric API path in pagination links.
const namedPrefix = TAG_ENDPOINT + '?';
const numericPrefix = 'https://api.github.com/repositories/20929025/tags?';
let query;
if (url.startsWith(namedPrefix)) {
query = url.slice(namedPrefix.length);
} else if (url.startsWith(numericPrefix)) {
query = url.slice(numericPrefix.length);
} else {
throw new Error('Refusing a page URL outside the expected GitHub repository tags endpoint.');
}
const parameters = query.split('&');
if (parameters.length !== 2) throw new Error('Refusing unexpected pagination query parameters.');
/** @type {string|null} */
let page = null;
/** @type {string|null} */
let pageSize = null;
for (const parameter of parameters) {
const pair = parameter.split('=');
if (pair.length !== 2) throw new Error('Refusing a malformed pagination query.');
if (pair[0] === 'page' && page === null) {
page = pair[1];
} else if (pair[0] === 'per_page' && pageSize === null) {
pageSize = pair[1];
} else {
throw new Error('Refusing unknown or repeated pagination parameters.');
}
}
if (page !== String(expectedPage) || pageSize !== String(TAG_PAGE_SIZE)) {
throw new Error('Refusing a repeated, skipped or resized next page.');
}
}
/**
* @param {Record<string, string|string[]>} headers
* @param {string} wantedName
* @returns {string}
*/
function tagHeader(headers, wantedName) {
for (const name of Object.keys(headers)) {
if (name.toLowerCase() === wantedName) {
const value = headers[name];
if (Array.isArray(value)) return value.join(', ');
return String(value);
}
}
return '';
}
/**
* @param {Record<string, string|string[]>} headers
* @returns {string}
*/
function tagWaitDetails(headers) {
const details = {
retryAfterSeconds: tagHeader(headers, 'retry-after').slice(0, 80),
remaining: tagHeader(headers, 'x-ratelimit-remaining').slice(0, 80),
resetEpochSeconds: tagHeader(headers, 'x-ratelimit-reset').slice(0, 80)
};
return 'Rate headers: ' + JSON.stringify(details) +
'. Honor Retry-After and any exhausted-budget reset before another run. ' +
'For a secondary limit without those headers, wait at least one minute; do not repeatedly rerun. ' +
'A 403 can also require a permission or policy fix.';
}
{
"timeZone": "Etc/UTC",
"dependencies": {},
"exceptionLogging": "STACKDRIVER",
"runtimeVersion": "V8",
"oauthScopes": [
"https://www.googleapis.com/auth/script.external_request"
]
}
Create a new standalone project named
Public tag pages.Replace
Code.gswith the pagination example file, replaceappsscript.jsonwith its included manifest, and save both files.Keep
TAG_MAX_PAGES = 2,TAG_MAX_ITEMS = 10, andTAG_PAGE_SIZE = 5.Select
listPublicRepositoryTagsand click Run. If prompted, approve thescript.external_requestscope.In the Execution log, find the page count, HTTP attempt count, collected tag count, result status, and tag names.
When two full pages succeed on the first attempt and GitHub has more pages, the log includes:
Repository: microsoft/TypeScript
Pages accepted: 2; HTTP attempts: 2
Tags collected: 10
Result: partial; page limit reached; more pages available The log then prints Tag names: and a JSON array of the 10 tag names. The run stopped at TAG_MAX_PAGES = 2 while a next link was still present, so the summary says partial. The 10 tags are not the repository's full list.
Follow the next link without following arbitrary URLs
GitHub's response carries a Link header with pagination URLs, including one marked rel="next" (GitHub pagination guide). The script does not build page URLs itself. It reads the rel="next" URL from the Link header and checks it before fetching the next page:
The URL must start with
https://api.github.com/repos/microsoft/TypeScript/tags?or with the repository's numeric pathhttps://api.github.com/repositories/20929025/tags?.The query string may hold only
pageandper_page, withper_page=5andpageequal to the next page number.
GitHub recommends that general clients follow redirects. This script disables them, so a moved or renamed repository stops the run for you to review.
Limit pages, items, and retries separately
For each page, the script requires HTTP 200, a JSON content type, a body of at most 100,000 characters, and a JSON array of at most five objects. Every item must have a non-empty tag name and a 40-character hex commit SHA.
TAG_MAX_PAGES limits the number of accepted pages, and TAG_MAX_ITEMS limits the number of tags kept. When TAG_MAX_ITEMS is reached partway through a page, both the inner for...of loop and the outer loop stop with break, so listPublicRepositoryTags can log its summary.
In the progress object, httpAttempts goes up before every UrlFetchApp.fetch call, retries included. pagesRead goes up only after a page passes its checks. With at most three attempts per page and TAG_MAX_PAGES = 2, a run makes at most six HTTP requests.
These counters limit one run. Apps Script separately enforces execution time, URL Fetch call, and response size quotas.
Retry only transient server errors
The forecast script makes one attempt. The GitHub helper fetchTagPage instead retries a transient server error (500, 502, 503, or 504) when no rate-limit signal is present. It makes up to three attempts per page, waiting 1 second before the second attempt and 2 seconds before the third.
| Result | What the script does |
|---|---|
fetch throws without an HTTP response | Stop and rethrow. The cause may be missing consent, a DNS or TLS failure, or an Apps Script quota. |
400, 401, 404, 410, or another client error | Stop at once. Repeating an unchanged request does not fix a client error. |
403 or 429 | Stop without a retry. A 403 can mean a rate limit or an access refusal. |
A Retry-After header, or x-ratelimit-remaining: 0 | Stop and log the rate-limit headers instead of sleeping through a long reset window. |
500, 502, 503, or 504 with requests remaining in the rate limit | Retry the GET request, up to three attempts in total, with 1 s and 2 s delays. |
200 with invalid JSON, an invalid item, or an invalid Link header | Stop and reject the page. |
GitHub's rate-limit documentation says to wait for Retry-After seconds when that header is present, or until x-ratelimit-reset (UTC epoch seconds) when x-ratelimit-remaining reaches 0. If an accepted page reports x-ratelimit-remaining: 0 and another page is wanted, the script stops with an error instead of requesting it.
Only a GET read that is safe to repeat belongs in this retry loop. Do not wrap a POST request, an email send, or a file creation in a retry loop that does not check whether the first attempt worked. If the network drops after the remote server has processed a write, a retry repeats the action.
Change the page limit and interpret a partial result
Change
TAG_MAX_PAGESfrom2to1, keepingTAG_MAX_ITEMS = 10andTAG_PAGE_SIZE = 5.Save and run
listPublicRepositoryTags. With a full first page and no retry, the log showsPages accepted: 1; HTTP attempts: 1,Tags collected: 5, andResult: partial; page limit reached; more pages available.Restore
TAG_MAX_PAGES = 2and save.
To test the item limit inside a page, keep TAG_MAX_PAGES = 2, set TAG_MAX_ITEMS = 3, and run once. The first five-item page passes its checks, the script keeps three tags, and the log reports Result: partial; item limit reached within a validated page. Restore TAG_MAX_ITEMS = 10 afterward.
Even a run that reaches the last page (complete) is a snapshot. Tags can be created or deleted between page requests, and those changes shift which tags fall on each page (GitHub pagination guide).
Restore the example settings
Restore FORECAST_DAYS = 7 in Forecast practice, and TAG_MAX_PAGES = 2 and TAG_MAX_ITEMS = 10 in Public tag pages. If you keep any forecast data, keep the Open-Meteo attribution rows in A9:B11 with it.
Use State across runs with PropertiesService to save configuration and avoid repeating network calls.
Project files
Complete source files for this chapter's examples.
Preview an HTTP request without sending it
Code.gs
Download filefunction previewRequest() {
const baseUrl = 'https://example.com/api/requests';
const query = 'status=open&limit=10';
const request = UrlFetchApp.getRequest(baseUrl + '?' + query, {
headers: { Accept: 'application/json' },
method: 'get',
});
Logger.log(JSON.stringify(request, null, 2));
}
appsscript.json
Download file{
"timeZone": "Etc/UTC",
"runtimeVersion": "V8",
"oauthScopes": [
"https://www.googleapis.com/auth/script.external_request"
]
}
sample-request.json
Download file{
"baseUrl": "https://example.com/api/requests",
"query": {
"status": "open",
"limit": 10
},
"headers": {
"Accept": "application/json"
}
}
Import a bounded public forecast into Sheets
Code.gs
Download fileconst FORECAST_DAYS = 7;
const FORECAST_LATITUDE = 52.52;
const FORECAST_LONGITUDE = 13.41;
const FORECAST_OWNER_MARKER = 'Owned by weather-import';
/**
* A property-access view for validation, not a validated provider response.
* Every data field remains unknown until the checks in forecastRows succeed.
* @typedef {{daily?: {time?: unknown, temperature_2m_max?: unknown, temperature_2m_min?: unknown}|null, daily_units?: {temperature_2m_max?: unknown, temperature_2m_min?: unknown}|null}} ForecastCandidate
*/
function importForecastToSheet() {
if (!Number.isInteger(FORECAST_DAYS) || FORECAST_DAYS < 1 || FORECAST_DAYS > 7) {
throw new Error('Set FORECAST_DAYS to a whole number from 1 to 7.');
}
if (!Number.isFinite(FORECAST_LATITUDE) || Math.abs(FORECAST_LATITUDE) > 90 ||
!Number.isFinite(FORECAST_LONGITUDE) || Math.abs(FORECAST_LONGITUDE) > 180) {
throw new Error('Use numeric latitude from -90 to 90 and longitude from -180 to 180.');
}
const spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
if (!spreadsheet) {
throw new Error('Open this project through Extensions > Apps Script in the practice spreadsheet.');
}
const sheet = spreadsheet.getSheetByName('Forecast');
if (!sheet) {
throw new Error('Create a blank tab named Forecast before running this function.');
}
const target = sheet.getRange('A1:C12');
checkForecastDestination(target, sheet);
const query = [
'latitude=' + encodeURIComponent(FORECAST_LATITUDE),
'longitude=' + encodeURIComponent(FORECAST_LONGITUDE),
'daily=temperature_2m_max,temperature_2m_min',
'temperature_unit=celsius',
'timezone=auto',
'forecast_days=' + FORECAST_DAYS
].join('&');
const response = UrlFetchApp.fetch('https://api.open-meteo.com/v1/forecast?' + query, {
method: 'get',
headers: { Accept: 'application/json' },
muteHttpExceptions: true,
followRedirects: false
});
const statusCode = response.getResponseCode();
if (statusCode !== 200) {
throw new Error('The forecast service answered with status code ' + statusCode + '. No cells were written.');
}
const contentType = forecastHeader(response.getAllHeaders(), 'content-type');
if (contentType.split(';')[0].trim().toLowerCase() !== 'application/json') {
throw new Error('The forecast response was not application/json. No cells were written.');
}
const text = response.getContentText('UTF-8');
if (text.length > 50000) {
throw new Error('The forecast response exceeded the 50000-character parsing limit.');
}
/** @type {unknown} */
let forecast;
try {
forecast = JSON.parse(text);
} catch (error) {
throw new Error('The forecast response was not valid JSON. No cells were written.');
}
const rows = forecastRows(forecast, FORECAST_DAYS);
// Check again after the request in case the destination changed while it was pending.
checkForecastDestination(target, sheet);
target.setValues(rows);
Logger.log('Wrote ' + FORECAST_DAYS + ' days of forecast.');
}
/**
* @param {SpreadsheetApp.Range} target
* @param {SpreadsheetApp.Sheet} sheet
* @returns {void}
*/
function checkForecastDestination(target, sheet) {
if (target.isPartOfMerge()) {
throw new Error('Forecast!A1:C12 must not overlap merged cells.');
}
if (target.isBlank()) return;
const markerCell = sheet.getRange('A12');
if (markerCell.getValue() !== FORECAST_OWNER_MARKER || markerCell.getFormula() !== '') {
throw new Error('Forecast!A1:C12 is not blank and has no valid ownership marker. Use a new practice sheet; do not clear existing data to bypass this check.');
}
}
/**
* @param {unknown} rawForecast
* @param {number} expectedDays
* @returns {Array<Array<string|number>>}
*/
function forecastRows(rawForecast, expectedDays) {
// This view permits property probes; the original runtime checks still validate the value.
const forecast = /** @type {ForecastCandidate|null|undefined} */ (rawForecast);
if (!forecast || typeof forecast !== 'object' || Array.isArray(forecast) ||
!forecast.daily || !forecast.daily_units) {
throw new Error('The forecast response is missing daily data or units.');
}
const days = forecast.daily.time;
const highs = forecast.daily.temperature_2m_max;
const lows = forecast.daily.temperature_2m_min;
if (!Array.isArray(days) || !Array.isArray(highs) || !Array.isArray(lows) ||
days.length !== expectedDays || highs.length !== expectedDays || lows.length !== expectedDays) {
throw new Error('The forecast arrays must each contain exactly ' + expectedDays + ' days.');
}
if (forecast.daily_units.temperature_2m_max !== '°C' ||
forecast.daily_units.temperature_2m_min !== '°C') {
throw new Error('The forecast temperature units must be Celsius.');
}
/** @type {Array<Array<string|number>>} */
const rows = [['Date', 'High', 'Low']];
let index = 0;
/** @type {number|null} */
let previousDay = null;
while (index < expectedDays) {
/** @type {unknown} */
const day = days[index];
/** @type {unknown} */
const high = highs[index];
/** @type {unknown} */
const low = lows[index];
if (typeof day !== 'string' || !/^\d{4}-\d{2}-\d{2}$/.test(day)) {
throw new Error('Forecast day ' + (index + 1) + ' is not a YYYY-MM-DD date.');
}
const timestamp = Date.parse(day + 'T00:00:00Z');
if (!Number.isFinite(timestamp) || new Date(timestamp).toISOString().slice(0, 10) !== day ||
(previousDay !== null && timestamp - previousDay !== 86400000)) {
throw new Error('Forecast dates must be valid consecutive calendar days.');
}
if (!Number.isFinite(high) || !Number.isFinite(low) ||
/** @type {number} */ (high) < /** @type {number} */ (low)) {
throw new Error('Forecast day ' + (index + 1) + ' needs numeric high and low temperatures with high >= low.');
}
rows.push([day, /** @type {number} */ (high), /** @type {number} */ (low)]);
previousDay = timestamp;
index = index + 1;
}
// The fixed output area has room for seven days, even when fewer are requested.
while (rows.length < 8) {
rows.push(['', '', '']);
}
rows.push(['Weather data by Open-Meteo', 'https://open-meteo.com/', '']);
rows.push(['Licence: CC BY 4.0', 'https://creativecommons.org/licenses/by/4.0/', '']);
rows.push(['Selected daily highs/lows; Celsius; local dates', '', '']);
rows.push([FORECAST_OWNER_MARKER, '', '']);
return rows;
}
/**
* @param {Record<string, string|string[]>} headers
* @param {string} wantedName
* @returns {string}
*/
function forecastHeader(headers, wantedName) {
for (const name of Object.keys(headers)) {
if (name.toLowerCase() === wantedName) {
const value = headers[name];
if (Array.isArray(value)) return value.join(', ');
return String(value);
}
}
return '';
}
appsscript.json
Download file{
"timeZone": "Etc/UTC",
"dependencies": {},
"exceptionLogging": "STACKDRIVER",
"runtimeVersion": "V8",
"oauthScopes": [
"https://www.googleapis.com/auth/script.external_request",
"https://www.googleapis.com/auth/spreadsheets.currentonly"
]
}
Read a bounded sample of public GitHub tags
Code.gs
Download fileconst TAG_MAX_PAGES = 2;
const TAG_MAX_ITEMS = 10;
const TAG_PAGE_SIZE = 5;
const TAG_ENDPOINT = 'https://api.github.com/repos/microsoft/TypeScript/tags';
/** @typedef {{name: string, commit: {sha: string}}} RepositoryTag */
/**
* A property-access view only; name and SHA are unknown until readTagPage validates them.
* @typedef {{name?: unknown, commit?: {sha?: unknown}|null}|null|undefined} TagCandidate
*/
function listPublicRepositoryTags() {
if (!Number.isInteger(TAG_MAX_PAGES) || TAG_MAX_PAGES < 1 || TAG_MAX_PAGES > 3 ||
!Number.isInteger(TAG_MAX_ITEMS) || TAG_MAX_ITEMS < 1 || TAG_MAX_ITEMS > 15 ||
TAG_PAGE_SIZE !== 5) {
throw new Error('Use 1-3 pages, 1-15 items, and a page size of 5.');
}
const progress = { httpAttempts: 0 };
const tags = [];
let pagesRead = 0;
/** @type {string|null} */
let url = TAG_ENDPOINT + '?per_page=' + TAG_PAGE_SIZE + '&page=1';
let complete = false;
let stopReason = '';
/** @type {unknown} */
let failure = null;
try {
while (url !== null) {
if (pagesRead >= TAG_MAX_PAGES) {
stopReason = 'page limit reached; more pages available';
break;
}
if (tags.length >= TAG_MAX_ITEMS) {
stopReason = 'item limit reached; more data available';
break;
}
checkTagPageUrl(url, pagesRead + 1);
const response = fetchTagPage(url, progress);
const headers = response.getAllHeaders();
const page = readTagPage(response, headers);
const nextUrl = nextTagPageUrl(headers);
if (nextUrl !== null) checkTagPageUrl(nextUrl, pagesRead + 2);
const room = TAG_MAX_ITEMS - tags.length;
for (const tag of page) {
if (tags.length >= TAG_MAX_ITEMS) break;
tags.push({ name: tag.name, sha: tag.commit.sha });
}
pagesRead = pagesRead + 1;
if (page.length > room) {
stopReason = 'item limit reached within a validated page';
break;
}
url = nextUrl;
if (url === null) {
complete = true;
stopReason = 'no next link reported';
} else if (pagesRead < TAG_MAX_PAGES && tags.length < TAG_MAX_ITEMS &&
tagHeader(headers, 'x-ratelimit-remaining') === '0') {
throw new Error('GitHub reports no requests remaining. ' + tagWaitDetails(headers));
}
}
} catch (error) {
failure = error;
stopReason = String(error);
}
let resultLabel = 'partial';
if (complete) resultLabel = 'complete';
Logger.log('Repository: microsoft/TypeScript');
Logger.log('Pages accepted: ' + pagesRead + '; HTTP attempts: ' + progress.httpAttempts);
Logger.log('Tags collected: ' + tags.length);
Logger.log('Result: ' + resultLabel + '; ' + stopReason);
Logger.log('Tag names: ' + JSON.stringify(tags.map((tag) => tag.name)));
if (failure !== null) throw failure;
}
/**
* @param {string} url
* @param {{httpAttempts: number}} progress
* @returns {UrlFetchApp.HTTPResponse}
*/
function fetchTagPage(url, progress) {
let attempt = 0;
while (attempt < 3) {
attempt = attempt + 1;
progress.httpAttempts = progress.httpAttempts + 1;
// Exceptions without an HTTP response are not automatically retried.
const response = UrlFetchApp.fetch(url, {
method: 'get',
headers: {
Accept: 'application/vnd.github+json',
'X-GitHub-Api-Version': '2026-03-10',
'User-Agent': 'apps-script-book-http-pagination'
},
muteHttpExceptions: true,
followRedirects: false
});
const status = response.getResponseCode();
const headers = response.getAllHeaders();
if (status === 200) return response;
if (status === 403 || status === 429 ||
tagHeader(headers, 'retry-after') !== '' ||
tagHeader(headers, 'x-ratelimit-remaining') === '0') {
throw new Error('GitHub HTTP ' + status + '; no automatic retry. ' + tagWaitDetails(headers));
}
const transient = status === 500 || status === 502 || status === 503 || status === 504;
if (!transient || attempt === 3) {
throw new Error('GitHub HTTP ' + status + ' after ' + attempt + ' attempt(s) for this page. Requests stopped.');
}
// Only these read-only GET failures are retried: wait 1 second, then 2 seconds.
Utilities.sleep(attempt * 1000);
}
throw new Error('The page attempt limit was reached.');
}
/**
* @param {UrlFetchApp.HTTPResponse} response
* @param {Record<string, string|string[]>} headers
* @returns {RepositoryTag[]}
*/
function readTagPage(response, headers) {
const contentType = tagHeader(headers, 'content-type').split(';')[0].trim().toLowerCase();
if (contentType !== 'application/json' && contentType !== 'application/vnd.github+json') {
throw new Error('GitHub returned an unexpected content type.');
}
const text = response.getContentText('UTF-8');
if (text.length > 100000) {
throw new Error('The GitHub response exceeded the 100000-character parsing limit.');
}
/** @type {unknown} */
let page;
try {
page = JSON.parse(text);
} catch (error) {
throw new Error('GitHub returned invalid JSON.');
}
if (!Array.isArray(page) || page.length > TAG_PAGE_SIZE) {
throw new Error('Expected an array containing at most five GitHub tags.');
}
for (const tag of /** @type {TagCandidate[]} */ (page)) {
if (!tag || typeof tag !== 'object' || Array.isArray(tag) ||
typeof tag.name !== 'string' || tag.name.length === 0 || tag.name.length > 256 ||
!tag.commit || typeof tag.commit.sha !== 'string' ||
!/^[0-9a-f]{40}$/.test(tag.commit.sha)) {
throw new Error('A GitHub tag is missing its expected name or commit SHA.');
}
}
return /** @type {RepositoryTag[]} */ (page);
}
/**
* @param {Record<string, string|string[]>} headers
* @returns {string|null}
*/
function nextTagPageUrl(headers) {
const link = tagHeader(headers, 'link');
if (link === '') return null;
if (link.length > 8000) throw new Error('GitHub pagination headers exceeded this example\'s limit.');
/** @type {string|null} */
let next = null;
for (const entry of link.split(',')) {
const match = entry.trim().match(/^<([^<>]+)>;\s*rel="(next|prev|first|last)"$/);
if (!match) throw new Error('GitHub returned an unsupported pagination link format.');
if (match[2] === 'next') {
if (next !== null) throw new Error('GitHub returned more than one next-page link.');
next = match[1];
}
}
return next;
}
/**
* @param {string} url
* @param {number} expectedPage
* @returns {void}
*/
function checkTagPageUrl(url, expectedPage) {
// GitHub may use this repository's numeric API path in pagination links.
const namedPrefix = TAG_ENDPOINT + '?';
const numericPrefix = 'https://api.github.com/repositories/20929025/tags?';
let query;
if (url.startsWith(namedPrefix)) {
query = url.slice(namedPrefix.length);
} else if (url.startsWith(numericPrefix)) {
query = url.slice(numericPrefix.length);
} else {
throw new Error('Refusing a page URL outside the expected GitHub repository tags endpoint.');
}
const parameters = query.split('&');
if (parameters.length !== 2) throw new Error('Refusing unexpected pagination query parameters.');
/** @type {string|null} */
let page = null;
/** @type {string|null} */
let pageSize = null;
for (const parameter of parameters) {
const pair = parameter.split('=');
if (pair.length !== 2) throw new Error('Refusing a malformed pagination query.');
if (pair[0] === 'page' && page === null) {
page = pair[1];
} else if (pair[0] === 'per_page' && pageSize === null) {
pageSize = pair[1];
} else {
throw new Error('Refusing unknown or repeated pagination parameters.');
}
}
if (page !== String(expectedPage) || pageSize !== String(TAG_PAGE_SIZE)) {
throw new Error('Refusing a repeated, skipped or resized next page.');
}
}
/**
* @param {Record<string, string|string[]>} headers
* @param {string} wantedName
* @returns {string}
*/
function tagHeader(headers, wantedName) {
for (const name of Object.keys(headers)) {
if (name.toLowerCase() === wantedName) {
const value = headers[name];
if (Array.isArray(value)) return value.join(', ');
return String(value);
}
}
return '';
}
/**
* @param {Record<string, string|string[]>} headers
* @returns {string}
*/
function tagWaitDetails(headers) {
const details = {
retryAfterSeconds: tagHeader(headers, 'retry-after').slice(0, 80),
remaining: tagHeader(headers, 'x-ratelimit-remaining').slice(0, 80),
resetEpochSeconds: tagHeader(headers, 'x-ratelimit-reset').slice(0, 80)
};
return 'Rate headers: ' + JSON.stringify(details) +
'. Honor Retry-After and any exhausted-budget reset before another run. ' +
'For a secondary limit without those headers, wait at least one minute; do not repeatedly rerun. ' +
'A 403 can also require a permission or policy fix.';
}
appsscript.json
Download file{
"timeZone": "Etc/UTC",
"dependencies": {},
"exceptionLogging": "STACKDRIVER",
"runtimeVersion": "V8",
"oauthScopes": [
"https://www.googleapis.com/auth/script.external_request"
]
}