Upgrading and reproducing results

This guide covers the cumulative impact of 2.0.19 → 2.2.5: find which changes touch your strategy, then compare identical input data, signals, positions, trades and reports before and after the upgrade, before switching production. Pinning FinLab alone does not pin pandas behavior or later data corrections, so the second half of the page shows how to save the environment, data and settings too.

How to upgrade

Which results change

Financial statements and quarterly data

Backtests and positions

Factor analysis, indicators and weights

Live order targets

These two change only the targets you order; backtest P&L and trades are unchanged.

Exact reproduction of old bugs requires the old wheel, identical dependencies and identical inputs; most fixes have no compatibility switch.

Quarterly Boolean masks: dtype and unknown values

The quarterly Boolean fix starts in 2.0.21 and remains in 2.2.16. A bool mask produced by comparing quarterly numbers, or a native nullable boolean mask, uses nullable boolean after index_str_to_date(): undisclosed values remain pd.NA, including after inversion with ~. Combining it with an ordinary daily bool mask using & produces ordinary bool, treating unknown values as False.

Inversion raises TypeError

The fix does not automatically convert object masks back to Boolean. Applying shift(1) first can introduce NaN and change the mask to object; date conversion keeps object, and ~ can still raise TypeError: bad operand type for unary ~: 'float'. Check mask.dtypes; for a mask holding only Boolean and missing values, convert to nullable boolean before date conversion:

mask = (q > 6).shift(1)  # q contains quarterly numbers; this mask is object
dated = mask.astype('boolean').index_str_to_date()
inverted = ~dated  # Unknown values remain pd.NA

You need plain bool or np.where

For a mask containing pd.NA, direct astype(bool) raises ValueError and direct np.where raises TypeError. Fill only if the strategy intends to treat unknown values as False, and note that the order of filling and inverting changes which stocks are selected:

import numpy as np

m = (q > 6).index_str_to_date()
selected = (~m).fillna(False)  # Invert first: undisclosed values stay unselected
opposite = ~m.fillna(False)  # Fill first: undisclosed values become True
plain_bool = selected.astype(bool)
weights = np.where(selected, 1, 0)

Download the cases and run them outside source checkouts: python upgrade_results.py quarter_bool_object reproduces the object inversion error; python upgrade_results.py quarter_bool_dtype demonstrates conversion and fill order. With the released PyPI finlab 2.2.16 wheel, pandas 3.0.6 and numpy 2.5.4, the restored dtype is boolean and inversion on August 2 gives [True, False]. Undisclosed B on May 1 remains pd.NA after inversion; inverting then filling gives False, while filling then inverting gives True. The original quarter_bool case still gives [True, False]; the error does not affect every 2.2.16 mask.

Defaults, output and exceptions

To keep cash after a stop, specify float weights before sim(): five stocks at 0.2 each is not the same as a Boolean basket, which redistributes equally. Exclude stocks with missing prices before computing the weights, and check position_limit.

Compare two versions with minimal cases

Download the cases. Each case calls only the affected calculation, with fixed prices, financial statements and filing dates. Quarters, the session calendar and the clock are synthetic fixtures, so the cases don't depend on later-corrected online data; calculations use the installed APIs and sim() uses the actual engine. Backtest cases require a normal FinLab login; see the Quickstart. These synthetic outputs locate differences between versions and do not represent real strategy performance.

Put the downloaded file outside source checkouts, create two environments and run the same case in each:

python3.13 -m venv before
python3.13 -m venv after
before/bin/python -m pip install "finlab==2.0.19" "pandas==2.2.3" "numpy==2.2.6" "TA-Lib==0.8.1" matplotlib
after/bin/python -m pip install "finlab==2.2.4" "pandas==2.2.3" "numpy==2.2.6" "TA-Lib==0.8.1" matplotlib
before/bin/python upgrade_results.py rank_ic
after/bin/python upgrade_results.py rank_ic

Replace rank_ic with any case below to compare the two versions.

The runs used macOS, CPython 3.13.3, pandas 2.2.3, numpy 2.2.6 and released PyPI FinLab wheels; TA-Lib 0.8.1 for the indicator case, a UTC host with a fixed clock for the date cases, and PYTHONHASHSEED=0 for duplicate columns. Decimals in the table are rounded; the full program prints the raw values. NaN means unknown or missing, and old exceptions are actual outputs.

Make results reproducible

Reproducing a backtest takes the same code, the same package versions, the same input data and the same sim() parameters; save all four.

Pin the environment

Keep production and comparison environments separate. Save the current environment first:

python -VV > python-version.txt
python -m pip freeze > requirements.lock.txt
python -m pip download --only-binary=:all: -r requirements.lock.txt -d wheels

Rebuild on the same Python version, operating system and CPU architecture:

python -m pip install --no-index --find-links=wheels -r requirements.lock.txt

Don't copy the version numbers of a verification environment onto an existing production one; pin the full dependency set production actually uses: FinLab, pandas, numpy, pyarrow, scipy, indicator packages (TA-Lib, pandas_ta) and model packages. For ML strategies, also save the models, random seeds and feature column order. Restart the Python or Jupyter kernel after upgrading, so old packages aren't still loaded in memory.

pandas versions change results too: in pandas 2.2, pct_change() still fills missing values forward by default, while pandas 3 uses fill_method=None. Choose the missing-value handling explicitly; see pandas 2.2 and pandas 3.

import pandas as pd

s = pd.Series([100.0, None, 110.0])
print(s.pct_change().tolist())
# pandas 2.2.3: [nan, 0.0, 0.10000000000000009]
# pandas 3.0.6: [nan, nan, nan]
print(s.pct_change(fill_method=None).tolist())  # Keep missing values explicitly
print(s.ffill().pct_change(fill_method=None).tolist())  # Fill first, explicitly

From 2.2.5, FinLab's built-in position.weight methods such as inverse_volatility(), event_study() and Position.from_weight() carry prices forward before computing returns, so pandas 2 and 3 agree; your own pct_change() calls still need an explicit choice.

Automatically save every strategy input

From 2.2.22, data.snapshot() saves the tables that data.get() and data.gets() actually return during one complete run to a new local directory, including financial statements and the prices, benchmarks and classifications read internally by sim(). Date ranges and explicit data versions are supported. Wrap the entire strategy from its first data read, and do the backtest and any report calculations inside the block:

from finlab import data
from finlab.backtest import sim

data.set_market('us')

def research():
    close = data.get('price:adj_close')
    position = (close > close.average(20)).loc['2020-01-01':]
    report = sim(
        position,
        resample='M',
        market='US_STOCK',
        end_date='2025-12-31',
        upload=False,
    )
    return report, report.get_stats()

# The directory must not already exist; the complete manifest is written only after the block exits successfully.
with data.snapshot('research-baseline'):
    baseline, baseline_stats = research()

# Keep the same code, parameters, date range, Python and package versions.
with data.snapshot('research-baseline', mode='replay'):
    restored, restored_stats = research()

# Normal reads resume outside the block. Save a new run to compare newer data.
with data.snapshot('research-new-data'):
    latest, latest_stats = research()

Worker threads in the same Python process take part in recording and replay, so wait for all parallel work to finish before leaving the block. Only one recording or replay block may be active at a time.

Replay reads only the saved inputs. Missing tables, unreadable files, or different read ranges or stock universes raise an error instead of falling back to the normal local cache or the latest cloud data. lazy=True is supported; the actual tables are saved before computation.

manifest.json records the dataset-to-file mapping and the Python, finlab, pandas, numpy and pyarrow versions; replay requires these versions to match. A snapshot preserves the inputs actually used in that run, not an atomic version of all tables at one instant. It cannot recover expired data that was never backed up, and it doesn't add new dates. Recording needs additional local disk space. Keep the entire directory, and load only trusted pickle files that you saved yourself.

These are not saved automatically:

• The strategy code, the full environment lock, random seeds, strategy parameters and a fixed end_date
• Variables populated before the block, custom prices or external data supplied by a custom Market, third-party APIs and other custom files: move the reads into the block or back them up separately

Historical replay does not mean a strategy is currently tradable. sim() still performs its normal login and authorization checks; set upload=False to avoid uploading replay results.

Record data versions by hand

From 2.2.0, paid members can list the generations still retained for a dataset with data.versions(), with publication and replacement times, size and hash, then download a chosen generation with data.get(dataset, version=...). Pick the generation first, then read it; don't fetch the latest data and attach metadata from a later request to it.

import json
from pathlib import Path
from finlab import data

data.set_market('us')
datasets = [
    'price:adj_close',
    'us_income_statement:net_income',
]  # every input of the strategy
manifest = {}
for dataset in datasets:
    versions = data.versions(dataset)
    if versions.empty:
        raise RuntimeError(f'{dataset} has no available version')
    row = versions.iloc[-1]
    generation = str(row['generation'])
    snapshot = data.get(dataset, version=generation)
    # Save the actual input; keep the file-to-dataset mapping in the manifest
    filename = f'input-{len(manifest)}.pkl'
    snapshot.to_pickle(filename)
    manifest[dataset] = {
        'generation': generation,
        'hash': str(row['hash']),
        'time_created': str(row['time_created']),
        'file': filename,
    }
Path('data-versions.json').write_text(json.dumps(manifest, indent=2))

Each generation pins one table, not an atomic snapshot across tables. Adjusted prices, filing dates, the trading calendar, universe membership and industry classifications are inputs too; save them as well. Keep saved data within your usage rights, and load only trusted pickles you saved yourself.

Save strategy and backtest settings

Save the strategy source, the execution instant and timezone, the environment lock, the data manifest, the processed input prices and the positions. Write down every sim() parameter instead of relying on defaults: market, trade price, rebalance frequency and offset, fees, taxes, stop loss and take profit, trailing stop, touched-exit mode, position limit, retain_cost_when_rebalance, stop_trading_next_period, start and end dates, and upload. For a custom Market, price matrix or benchmark, save its code and inputs, not just its name.

import json
from pathlib import Path
from finlab import data
from finlab.backtest import sim

data.set_market('us')
with data.us_universe(index='SP500'):
    close = data.get('price:adj_close')
position = (close / close.shift(126)).is_largest(
    20
)  # the position your strategy produces

settings = dict(
    resample='M',
    trade_at_price='close',
    market='US_STOCK',
    fee_ratio=0,
    tax_ratio=0,
    position_limit=1,
    stop_loss=None,
    take_profit=None,
    trail_stop=None,
    touched_exit=False,
    retain_cost_when_rebalance=False,
    stop_trading_next_period=True,
    end_date='2025-12-31',
    upload=False,
)
position.to_pickle('position.pkl')  # Save the position you actually passed
Path('sim-settings.json').write_text(json.dumps(settings, indent=2))
report = sim(position, **settings)
report.to_pickle('report.pkl')
report.to_html('report.html')

report.pkl, HTML or cloud reports save the results, but do not archive all source data, dependencies, source code or custom markets, and a backtest report is not a record of actual executed trades. To verify a reproduction, compare data values and dates, signals, positions, trade dates and quantities and the return curve, then summary metrics such as Sharpe. Fix end_date first, and validate the latest rebalance and live targets separately.

Track release impact in code

Public feeds: JSON and Atom. Both are generated when the docs are built, from the same changelog and reviewed, item-by-item impact annotations, and cover published releases from 2.0.19. Unreleased is excluded.

Each item has three Boolean flags (results_may_change, behavior_change, experimental), functions, trigger, references and the full description. results_may_change covers calculation, selection and backtest numbers; changes to live targets only use behavior_change, so read the description either way. False does not mean the data will never be corrected.

This lists every item after 2.0.19 and up to 2.2.5 flagged as possibly changing results or behavior:

import requests

version = lambda text: tuple(map(int, text.split('.')))
start, end = version('2.0.19'), version('2.2.5')
response = requests.get('https://finlab.finance/docs/releases.json', timeout=30)
response.raise_for_status()
for release in response.json()['releases']:
    if start < version(release['version']) <= end:
        for item in release['items']:
            if item['results_may_change'] or item['behavior_change']:
                print(release['version'], item['functions'], item['trigger'])

The range excludes the start version and includes the target; compare versions as numbers, not strings. The two yanked releases remain in the feed for readers who upgraded through them. Annotations record each release's change as it was, so watch for later reversions, such as hold_until in 2.2.1 and the cache check in 2.2.4. Docs and feeds are deployed and verified before the release is uploaded to PyPI.

Next steps