Master IBSS
Interactive Brokers' shortable-stock inventory across all 18 markets it publishes: shares available to borrow, borrow fee and rebate rate per name and session.
master_ibss — the dataset name to pass to the Obscura API.
What one row means
One instrument's shortable-share inventory entry from one Interactive Brokers shortstock file: the shares of that instrument IB had available to lend/short on one market, on one session, with the per-instrument rebate and fee rates and identifiers IB publishes alongside them. IB serves one file per market (`usa.txt`, `canada.txt`, `british.txt`, … — 18 of them), so the grain is the market as well as the instrument: 804 tickers appear in both `usa` and `canada`, 668 in both `usa` and `germany`, and without `country` in the key they overwrite each other. Table name matches rw-task-scheduler.
One row per (occurred_on, country, ticker).
Point-in-time availability
Every Obscura dataset carries available_date: the calendar day the publisher made the row
available, day-of, with no session rounding. It is the one column a backtest filters on, and it means the
same thing on every dataset in the catalog.
For master_ibss: published: available_date = the calendar day Interactive Brokers made the market's shortstock file available — the file's own `#BOF|YYYY.MM.DD` generation stamp when it carries one, else the publication day the file was captured under. It is NOT the session the inventory describes: IB's US file dated D carries the D-1 US session, so `occurred_on` is one US EQUITY business day behind `available_date` on `country = 'usa'` and equal to it on every other market.
Refresh cadence
Obscura refreshes master_ibss every weekday — the most frequent scheduled job that re-collects or re-exports it. This is Obscura's own pipeline cadence, not the upstream publisher's release schedule; when a row became public is recorded per row in available_date.
Schema — 12 columns
The full public column list for master_ibss, with the meaning of every field. The same
schema is served unauthenticated at https://api.obscura.trade/v1/catalog/master_ibss.
| Column | Type | Description |
|---|---|---|
| occurred_on | date · not null | The session this inventory describes — the reference implementation's `date` column. On `country = 'usa'` it is `available_date` minus exactly one US EQUITY business day (weekends and NYSE holidays skipped), because IB's US file dated D reports the D-1 session; on every other market IB's file describes the day it is dated, so this equals `available_date`. Never later than `available_date` — the session is always already over when the file naming it is published. First component of the natural key. |
| country | text · not null | Which of Interactive Brokers' per-market shortstock files this row came from, as IB names them: `usa`, `canada`, `british`, `dutch`, `germany`, `france`, `italy`, `spain`, `austria`, `belgium`, `swedish`, `swiss`, `japan`, `hongkong`, `australia`, `mexico`, `india`, `singapore`. These are IB's own file stems and NOT ISO country codes — several are adjectives (`british`, `dutch`) and one is a city (`hongkong`). Second component of the natural key: the same ticker trades on several of these markets on the same session with different borrow terms. |
| ticker | text · not null | IB's ticker symbol for the instrument on this market (source field `#SYM`), e.g. "AAPL", a CUSIP-like bond stem such as "049323AB4", or a suffixed variant like "AURC.OLD1" for a renamed/delisted issue. Third component of the natural key. |
| available_date | date · not null | PUBLIC-availability date: the calendar day IB made this market's file available. Read from the file's own `#BOF|YYYY.MM.DD|HH:MM:SS` header when it carries one (IB's statement of when it generated the file), else from the publication day the file was captured and landed under. This is the point-in-time column to filter and join on — NEVER `occurred_on`, which on `usa` is a session that was already over when this row became knowable. |
| currency | text | Currency the instrument is quoted in on this market (source field `#CUR`, e.g. "USD", "CAD", "EUR"). NULL if the field was empty or literally "NA". |
| company_name | text | Issuer/security display name as IB carries it (source field `#NAME`), typically uppercase and sometimes containing HTML entities (`&`) as delivered by the source. NULL if empty or "NA". |
| con | double precision | Interactive Brokers' internal contract identifier for the instrument (source field `#CON`). Held as a float, not an integer, because that is what the source emits in its comma-delimited generation — `1715006.0` — and the reference schema types it `Float` for the same reason. NULL if the field was blank or unparseable. |
| isin | text | ISIN of the instrument (source field `#ISIN`). NULL if empty or literally "NA". Values sometimes have their digits masked with 'X' by the source for restricted/renamed issues (`XXXXXXXU1016`). |
| rebate_rate | double precision | Annualized interest rebate rate (percent) IB pays/charges on cash collateral for borrowing this instrument (source field `#REBATERATE`). Negative for hard-to-borrow names. NULL if blank, "NA" or unparseable — IB's India file publishes every row with both rate fields empty. |
| fee_rate | double precision | Annualized borrow fee rate (percent) charged to short this instrument (source field `#FEERATE`) — higher values signal scarcer, harder-to-borrow shares. NULL if blank, "NA" or unparseable. |
| available | text | Shares available to short, VERBATIM as IB published them (source field `#AVAILABLE`). Deliberately TEXT, not a number: IB caps the figure with a bound rather than a value — `>10000000` means "more than ten million and we are not saying how many" — and no integer can represent that. Parsing it to `10000000` claimed a precision the publisher refused to give and made a capped name indistinguishable from one with exactly that many shares. IB's comma-delimited generation (2020-07-13 … 2024-01-16) also emits some values with a `.0` suffix; those are kept as published too. |
| scraped_at | timestamp with time zone · not null | Internal ingestion timestamp set when the orchestrator upserted this row. Row bookkeeping only, distinct from both `available_date` (when IB published) and `occurred_on` (which session); excluded from exports. |
Access master_ibss
Two delivery paths, one identifier. Both require an Obscura account and an active subscription; the catalog entry and the schema above are public.
import obscura
client = obscura.Client("obs_live_…")
df = client.query(
dataset="master_ibss",
symbols=["NVDA", "AAPL"],
start="2024-01-01",
)Create a free account Browse all 95 datasets
Frequently asked questions
What is in the master_ibss dataset?
Interactive Brokers' shortable-stock inventory across all 18 markets it publishes: shares available to borrow, borrow fee and rebate rate per name and session. One instrument's shortable-share inventory entry from one Interactive Brokers shortstock file: the shares of that instrument IB had available to lend/short on one market, on one session, with the per-instrument rebate and fee rates and identifiers IB publishes alongside them. IB serves one file per market (`usa.txt`, `canada.txt`, `british.txt`, … — 18 of them), so the grain is the market as well as the instrument: 804 tickers appear in both `usa` and `canada`, 668 in both `usa` and `germany`, and without `country` in the key they overwrite each other. Table name matches rw-task-scheduler.
How do I avoid look-ahead bias with master_ibss?
Filter on master_ibss.available_date, the day the publisher made the row public. For this dataset that date is derived as follows — published: available_date = the calendar day Interactive Brokers made the market's shortstock file available — the file's own `#BOF|YYYY.MM.DD` generation stamp when it carries one, else the publication day the file was captured under. It is NOT the session the inventory describes: IB's US file dated D carries the D-1 US session, so `occurred_on` is one US EQUITY business day behind `available_date` on `country = 'usa'` and equal to it on every other market. A query of the form WHERE available_date <= '<as-of date>' never sees a row before it existed.
In what formats can I get master_ibss?
As a Parquet bulk export (POST https://api.obscura.trade/v1/download) or as JSON from the typed query API (POST https://api.obscura.trade/v1/query), both with dataset="master_ibss". The column schema is public at https://api.obscura.trade/v1/catalog/master_ibss.
Can I filter master_ibss by company or symbol?
Yes. master_ibss carries ticker, the column the API's symbols filter resolves against.
How often is master_ibss updated?
Obscura refreshes master_ibss on a every weekday schedule — that is the most frequent scheduled job that re-collects or re-exports the table. It is Obscura's own pipeline cadence, not the upstream publisher's release schedule; when the publisher makes a row available is described by the availability rule above, and is recorded per row in available_date.