GitHub Repos
Activity on tracked GitHub repositories: stars, forks and commit frequency.
github_repos — the dataset name to pass to the Obscura API.
What one row means
One daily live-snapshot reading of a single tracked GitHub repository's public counters (stars, forks, open issues, watchers, primary language, last-push timestamp) as returned by the keyless GitHub REST API (api.github.com/repos/{owner}/{name}) at the moment the collector fetched it.
One row per (repo, captured_date).
Point-in-time availability
Every Obscura dataset carries available_date: the calendar day the publisher made the row
available, day-of, with no session rounding. It is the one column a backtest filters on, and it means the
same thing on every dataset in the catalog.
For github_repos: snapshot: available_date = captured_date, a STORED GENERATED column (migration 000205). WHY THAT IS THE DAY IT BECAME AVAILABLE: the /repos endpoint returns cumulative counters as they stand at the instant of the request and keeps no history, so a row for a past day cannot be fetched after the fact — the day the collector polled is the day this particular reading was obtainable. Note what this date is NOT: it is not the day the stars were earned. EVIDENCE captured_date is the capture day: Utc::now().date_naive() live, BronzeObject::captured_day() on replay. Generated rather than collector-written so no clock can reach it on a replay.
Refresh cadence
Obscura refreshes github_repos daily — the most frequent scheduled job that re-collects or re-exports it. This is Obscura's own pipeline cadence, not the upstream publisher's release schedule; when a row became public is recorded per row in available_date.
Schema — 14 columns
The full public column list for github_repos, with the meaning of every field. The same
schema is served unauthenticated at https://api.obscura.trade/v1/catalog/github_repos.
| Column | Type | Description |
|---|---|---|
| repo | text · not null | "owner/name" GitHub repo slug, e.g. "apache/spark" or "pytorch/pytorch" — the exact path segment used in the api.github.com/repos/{owner}/{name} request URL. Half of the composite primary key with available_date; comes from the operator-configured GithubArgs.repos list, not discovered from the API. |
| captured_date | date · not null | The calendar day (UTC) the collector POLLED the GitHub API — Utc::now().date_naive() once per run on a live run, the bronze object's capture day on a replay — not a source-provided date. Other half of the composite primary key; the upsert's OnConflict target is (repo, captured_date), so re-running the task the same day overwrites the same row rather than duplicating it. available_date is generated directly from this column, because the endpoint keeps no history and the day we asked is the only day this counter reading was obtainable. |
| stars | bigint | Repo star count at fetch time, from the API's stargazers_count field. Cumulative, monotonically non-decreasing counter (can dip slightly on unstars); nullable because parsing tolerates a missing/non-integer field. The primary open-source-attention signal for this dataset. |
| forks | bigint | Repo fork count at fetch time, from the API's forks_count field. Cumulative counter; nullable on missing field. |
| open_issues | bigint | Open issue count (issues + open PRs, per GitHub's API semantics) at fetch time, from the API's open_issues_count field. A point-in-time gauge, not cumulative — can rise and fall day to day. Nullable on missing field. |
| watchers | bigint | Watcher/subscriber count at fetch time. Prefers the API's subscribers_count field and falls back to watchers_count if subscribers_count is absent — GitHub's watchers_count field has historically been an alias for stargazers_count on some API versions, so subscribers_count is the more accurate "people watching activity" figure when present. Nullable on missing field. |
| language | text | GitHub's detected primary/dominant language for the repo at fetch time (API field `language`, e.g. "Python", "C++", "Scala"), or null if GitHub reports none (e.g. a docs-only or newly-created repo). Can change over time as the repo's dominant language shifts; not historized beyond this daily snapshot. |
| pushed_at | timestamp with time zone | The repo's own last-push timestamp as reported by the GitHub API's `pushed_at` field (RFC3339, converted to UTC) — a natural/state date describing repo activity, deliberately kept separate from available_date. Reflects when code was last pushed to the default branch, independent of when this platform observed it; nullable if the API omits or parsing fails. Never conflated with or used in place of available_date. |
| repo_id | bigint | GitHub's stable numeric repository id (API field `id`, e.g. 17165658). Unlike the `repo` slug PK, this survives owner/name renames, so it is the durable cross-rename identifier for a repo across snapshots and datasets. Nullable on missing/non-integer field. |
| created_at | timestamp with time zone | The repository's own creation timestamp as reported by the GitHub API's `created_at` field (RFC3339, converted to UTC). A distinct temporal signal (yields project age) not derivable from `pushed_at`/`available_date`; deliberately kept separate from available_date and never used in its place. Nullable if the API omits or parsing fails. |
| archived | boolean | Whether the repo is archived (API field `archived`): an archived repo is read-only / effectively abandoned. A status flag that materially reframes how the star/fork/issue counters should be read. Nullable on missing field. |
| size_kb | bigint | Repository size in kilobytes at fetch time (API field `size`) — a distinct codebase-size measure independent of the attention counters. Nullable on missing/non-integer field. |
| license_spdx_id | text | The repo's open-source license SPDX id (API field `license.spdx_id`, e.g. "Apache-2.0", "MIT", or "NOASSERTION"). A licensing classification dimension; null when the `license` object is absent. The genuinely 1:1 representative of the nested `license` block (`license.key` is a redundant lower-cased alias). |
| available_date | date | PUBLIC-availability date = captured_date. STORED GENERATED column (migration 000205), read-only; the point-in-time column to filter and join on. It is the day this platform polled GitHub, because the API serves only the current counter values and keeps no archive to fetch a past day from. |
Access github_repos
Two delivery paths, one identifier. Both require an Obscura account and an active subscription; the catalog entry and the schema above are public.
import obscura
client = obscura.Client("obs_live_…")
df = client.query(
dataset="github_repos",
symbols=["NVDA", "AAPL"],
start="2024-01-01",
)Create a free account Browse all 95 datasets
Frequently asked questions
What is in the github_repos dataset?
Activity on tracked GitHub repositories: stars, forks and commit frequency. One daily live-snapshot reading of a single tracked GitHub repository's public counters (stars, forks, open issues, watchers, primary language, last-push timestamp) as returned by the keyless GitHub REST API (api.github.com/repos/{owner}/{name}) at the moment the collector fetched it.
How do I avoid look-ahead bias with github_repos?
Filter on github_repos.available_date, the day the publisher made the row public. For this dataset that date is derived as follows — snapshot: available_date = captured_date, a STORED GENERATED column (migration 000205). WHY THAT IS THE DAY IT BECAME AVAILABLE: the /repos endpoint returns cumulative counters as they stand at the instant of the request and keeps no history, so a row for a past day cannot be fetched after the fact — the day the collector polled is the day this particular reading was obtainable. Note what this date is NOT: it is not the day the stars were earned. EVIDENCE captured_date is the capture day: Utc::now().date_naive() live, BronzeObject::captured_day() on replay. Generated rather than collector-written so no clock can reach it on a replay. A query of the form WHERE available_date <= '<as-of date>' never sees a row before it existed.
In what formats can I get github_repos?
As a Parquet bulk export (POST https://api.obscura.trade/v1/download) or as JSON from the typed query API (POST https://api.obscura.trade/v1/query), both with dataset="github_repos". The column schema is public at https://api.obscura.trade/v1/catalog/github_repos.
Can I filter github_repos by company or symbol?
Yes. github_repos carries repo, the column the API's symbols filter resolves against.
How often is github_repos updated?
Obscura refreshes github_repos on a daily schedule — that is the most frequent scheduled job that re-collects or re-exports the table. It is Obscura's own pipeline cadence, not the upstream publisher's release schedule; when the publisher makes a row available is described by the availability rule above, and is recorded per row in available_date.