Wikipedia Revisions
Edit activity on tracked Wikipedia articles.
wikipedia_revisions — the dataset name to pass to the Obscura API.
What one row means
One daily edit-activity snapshot for a single covered ticker's Wikipedia article: the number of MediaWiki revisions observed in the trailing window (default 30 days) as of the moment the task ran, plus the article's current byte size and the timestamp of its single most recent revision.
One row per (ticker, captured_date).
Point-in-time availability
Every Obscura dataset carries available_date: the calendar day the publisher made the row
available, day-of, with no session rounding. It is the one column a backtest filters on, and it means the
same thing on every dataset in the catalog.
For wikipedia_revisions: snapshot: available_date = captured_date, a STORED GENERATED column. WHY THAT IS THE DAY IT BECAME AVAILABLE: the row's measures are not source facts at all — the trailing revisions_window COUNT and the article_bytes reading are computed by this collector at query time against the live MediaWiki API, so they come into existence on the day it ran and on no earlier day. There is no publisher and no publication event to be early or late against. EVIDENCE captured_date is the capture day: Utc::now().date_naive() live, BronzeObject::captured_day() on replay. Renamed from `snapshot_date` by migration 000205 so every snapshot dataset names the act identically.
Refresh cadence
Obscura refreshes wikipedia_revisions every weekday — the most frequent scheduled job that re-collects or re-exports it. This is Obscura's own pipeline cadence, not the upstream publisher's release schedule; when a row became public is recorded per row in available_date.
Schema — 15 columns
The full public column list for wikipedia_revisions, with the meaning of every field. The same
schema is served unauthenticated at https://api.obscura.trade/v1/catalog/wikipedia_revisions.
| Column | Type | Description |
|---|---|---|
| ticker | text · not null | Stock ticker of the covered company, drawn from the ticker->article resolution cache. Part 1 of the composite primary key. |
| captured_date | date · not null | UTC calendar day the collector CAPTURED the article (`Utc::now().date_naive()` at the start of run(), the bronze object's capture day on a replay) — the day the trailing revisions_window count and article_bytes value were actually computed, and therefore the day they became knowable. Part 2 of the composite primary key, and the column available_date is generated from. Named `snapshot_date` before migration 000205. |
| article | text | Resolved MediaWiki page title for the ticker, used to build the query against en.wikipedia.org/w/api.php. NULL only for tickers the task skips entirely (rows written always carry a value). |
| pageid | bigint | Stable numeric MediaWiki page ID (raw `query.pages[0].pageid`, e.g. 856). A rename-proof canonical join key for the article that survives title changes, unlike the `article` title string. NULL if the API returned no page (missing/deleted title). |
| rev_id | bigint | ID of the article's current (newest) revision (raw `query.pages[0].revisions[0].revid`; rvprop=ids). Uniquely identifies the exact edit that produced the snapshot's article_bytes/event_time. NULL if zero revisions returned. |
| rev_parent_id | bigint | ID of the revision immediately preceding the current one (raw `query.pages[0].revisions[0].parentid`; rvprop=ids). 0 for a page's first revision. NULL if zero revisions returned. |
| rev_user | text | Editor of the current (newest) revision (raw `query.pages[0].revisions[0].user`; rvprop=user) — a username or an IP address for anonymous edits. NULL if zero revisions returned or the field is hidden/suppressed. |
| rev_user_id | bigint | Numeric MediaWiki user ID of the current revision's editor (raw `query.pages[0].revisions[0].userid`; rvprop=userid). 0 for anonymous/IP edits. NULL if zero revisions returned or the field is hidden. |
| rev_minor | boolean | Minor-edit flag for the current revision (raw `query.pages[0].revisions[0].minor`; rvprop=flags). True when the editor marked the edit minor; absent in the payload for non-minor edits, so NULL is read as "not minor / unknown". |
| rev_comment | text | Edit summary / comment attached to the current revision (raw `query.pages[0].revisions[0].comment`; rvprop=comment). Free text describing the edit; an edit-quality / attention signal. NULL if zero revisions, empty, or hidden. |
| rev_sha1 | text | SHA-1 content hash of the current revision (raw `query.pages[0].revisions[0].sha1`; rvprop=sha1). Lets a consumer detect real content changes vs. no-op edits between snapshots. NULL if zero revisions returned or the field is hidden. |
| revisions_window | integer · not null | Count of MediaWiki revisions to the article with a timestamp on/after (snapshot instant - window_days), window_days defaulting to 30. Capped at the API's rvlimit=500 per call — the task does not paginate, so a heavily-edited article within the window can be undercounted. |
| article_bytes | bigint | Byte size of the article's current (most recent) revision. NULL if the API returned zero revisions (e.g. the page was deleted, merged, or renamed since caching). |
| event_time | timestamp with time zone | Timestamp of the article's single most recent MediaWiki revision as of the snapshot. A genuine natural/occurred-on moment for the edit itself — explicitly NOT the row's public-availability date (see available_date). NULL when the API returned zero revisions. |
| available_date | date | PUBLIC-availability date. STORED generated column = `captured_date` — the UTC calendar day the collector computed this row's counts; the point-in-time column to filter/join on. DB-generated, read-only. |
Access wikipedia_revisions
Two delivery paths, one identifier. Both require an Obscura account and an active subscription; the catalog entry and the schema above are public.
import obscura
client = obscura.Client("obs_live_…")
df = client.query(
dataset="wikipedia_revisions",
symbols=["NVDA", "AAPL"],
start="2024-01-01",
)Create a free account Browse all 95 datasets
Frequently asked questions
What is in the wikipedia_revisions dataset?
Edit activity on tracked Wikipedia articles. One daily edit-activity snapshot for a single covered ticker's Wikipedia article: the number of MediaWiki revisions observed in the trailing window (default 30 days) as of the moment the task ran, plus the article's current byte size and the timestamp of its single most recent revision.
How do I avoid look-ahead bias with wikipedia_revisions?
Filter on wikipedia_revisions.available_date, the day the publisher made the row public. For this dataset that date is derived as follows — snapshot: available_date = captured_date, a STORED GENERATED column. WHY THAT IS THE DAY IT BECAME AVAILABLE: the row's measures are not source facts at all — the trailing revisions_window COUNT and the article_bytes reading are computed by this collector at query time against the live MediaWiki API, so they come into existence on the day it ran and on no earlier day. There is no publisher and no publication event to be early or late against. EVIDENCE captured_date is the capture day: Utc::now().date_naive() live, BronzeObject::captured_day() on replay. Renamed from `snapshot_date` by migration 000205 so every snapshot dataset names the act identically. A query of the form WHERE available_date <= '<as-of date>' never sees a row before it existed.
In what formats can I get wikipedia_revisions?
As a Parquet bulk export (POST https://api.obscura.trade/v1/download) or as JSON from the typed query API (POST https://api.obscura.trade/v1/query), both with dataset="wikipedia_revisions". The column schema is public at https://api.obscura.trade/v1/catalog/wikipedia_revisions.
Can I filter wikipedia_revisions by company or symbol?
Yes. wikipedia_revisions carries ticker, the column the API's symbols filter resolves against.
How often is wikipedia_revisions updated?
Obscura refreshes wikipedia_revisions on a every weekday schedule — that is the most frequent scheduled job that re-collects or re-exports the table. It is Obscura's own pipeline cadence, not the upstream publisher's release schedule; when the publisher makes a row available is described by the availability rule above, and is recorded per row in available_date.