API reference

finlab.ml

On this page8

finlab.ml gets your data into the shape machine learning needs: feature combines datasets into features, label builds training targets, and qlib provides ready-made models. A model's prediction is a table of dates by symbols, so you can pick stocks from it with is_largest() and backtest it with sim().

Which function to use

You want to	Use
Combine several datasets into one feature table	`mlf.combine()`
Compute TA-Lib indicators in bulk as features	`mlf.ta()`
List indicator names (optionally with random parameters)	`mlf.ta_names()`
Label each sample with its return over the next N periods	`mll.return_percentage()`
Label each sample with the next day's open-to-close return (day trading)	`mll.daytrading_percentage()`
Label each sample with its maximum adverse excursion over the next N periods (risk)	`mll.maximum_adverse_excursion()`
Train a model and predict	`q.LGBModel()`, `q.XGBModel()`, `q.CatBoostModel()`, `q.LinearModel()`

The examples use these aliases: from finlab.ml import feature as mlf, label as mll and import finlab.ml.qlib as q.

From features to a backtest

The whole flow of an ML stock-selection strategy: build features → build labels → split train and test by date → train → predict → turn predictions into positions → backtest.

from finlab import data
from finlab.ml import feature as mlf, label as mll
import finlab.ml.qlib as q
from finlab.backtest import sim

data.set_market('us')
with data.us_universe(index='SP500'):
    raw_close = data.get('price:close')  # unadjusted price for valuation
    market_value = data.get('etl:us_market_value')
    eps = data.get('us_income_statement:eps_diluted')
    net_income = data.get('us_income_statement:net_income')
    equity = data.get('us_balance_sheet:total_stockholders_equity')
    # Technical indicators (use few to avoid overfitting)
    technical = mlf.ta(mlf.ta_names(n=1)[:20])  # Only the first 20

# Step 1: Build feature set
print("Building features...")
ttm_eps = eps.rolling(4).sum()
features = mlf.combine(
    {
        # Fundamental features, only where the denominator is positive
        'pb': market_value / equity[equity > 0],
        'pe': raw_close / ttm_eps[ttm_eps > 0],  # Trailing-12-month P/E
        'roe': net_income / equity[equity > 0],
        'technical': technical,
    },
    resample='W',
)

# Step 2: Generate labels
print("Generating labels...")
label = mll.return_percentage(features.index, resample='W', period=1)

# Step 3: Split train/test sets
print("Splitting data...")
is_train = features.index.get_level_values('datetime') < '2023-01-01'
X_train, y_train = features[is_train], label[is_train]
X_test = features[~is_train]  # LightGBM handles missing values itself

# Step 4: Train model
print("Training model...")
model = q.LGBModel()
model.fit(X_train, y_train)

# Step 5: Predict
print("Generating predictions...")
pred = model.predict(X_test)

# Step 6: Convert to trading signals
print("Generating trading signals...")
position = pred.is_largest(30)  # Buy stocks with highest predicted returns

# Step 7: Backtest
print("Running backtest...")
report = sim(position, resample='W')
report.display()

Combine features

mlf.combine()

Combines a dictionary of features into one DataFrame indexed by (datetime, instrument), with one column per feature. Wide features (dates by symbols) are reindexed onto the union of their dates and the symbols they share, forward-filled, then stacked; features that already have a (datetime, instrument) MultiIndex are concatenated as they are. Quarterly FinLab data is converted to filing dates first.

from finlab import data
from finlab.ml import feature as mlf

data.set_market('us')
with data.us_universe(index='SP500'):
    close = data.get('price:adj_close')
    raw_close = data.get('price:close')  # unadjusted price for valuation
    market_value = data.get('etl:us_market_value')
    eps = data.get('us_income_statement:eps_diluted')
    net_income = data.get('us_income_statement:net_income')
    equity = data.get('us_balance_sheet:total_stockholders_equity')

ttm_eps = eps.rolling(4).sum()
pb = market_value / equity[equity > 0]
pe = raw_close / ttm_eps[ttm_eps > 0]  # Trailing-12-month P/E

# Example 1: Combine fundamental features
features = mlf.combine(
    {
        'pb': pb,
        'pe': pe,
        'roe': net_income / equity[equity > 0],
    },
    resample='W',
)

# Example 2: Combine technical indicators
indicators = [
    'talib.RSI__real__timeperiod__14__',
    'talib.MACD__macd__fastperiod__12__slowperiod__26__signalperiod__9__',
]
with data.us_universe(index='SP500'):
    features = mlf.combine({'talib': mlf.ta(indicators)}, resample='D')

# Example 3: Mix multiple feature types
with data.us_universe(index='SP500'):
    technical = mlf.ta(indicators)
features = mlf.combine(
    {
        'fundamental': mlf.combine({'pb': pb, 'pe': pe}),
        'technical': technical,
        'custom': close / close.average(20),  # Price relative to its 20-day average
    },
    resample='W',
)

Parameter	Type	Default	Description
`features`	`dict[str, pd.DataFrame or Callable]`	required	Feature name to a DataFrame (dates by symbols, or a MultiIndex frame), or a function returning one.
`resample`	`str`, `pd.DatetimeIndex` or `None`	`None`	Optional resampling; `None` keeps the original dates. MultiIndex features use the same resampling.
`sample_filter`	`pd.DataFrame` or `None`	`None`	Boolean DataFrame (dates by symbols) selecting which samples to keep.
`**kwargs`			Passed to the resampler.

Returns a pd.DataFrame with all input features combined.

resample can be:

'D': daily
'W': weekly (string frequencies use calendar period labels, so the week is labeled on its calendar end, Sunday)
'M': monthly (month-end)
a pd.DatetimeIndex: explicit date anchors, for example each week's last trading day

To anchor features on each week's last trading day and build labels on the same dates, pass the same dates to both:

import pandas as pd
from finlab import data
from finlab.ml import feature as mlf, label as mll

data.set_market('us')
rsi = data.indicator('RSI')
calendar = rsi.index
weekly = pd.DatetimeIndex(calendar.to_series().resample('W').last().dropna())
features = mlf.combine({'rsi': rsi}, resample=weekly)
labels = mll.return_percentage(features.index, resample=weekly, period=2)

Compute technical indicators

mlf.ta()

Computes TA-Lib indicators in bulk from their names, which look like talib.RSI__real__timeperiod__14__.

from finlab import data
from finlab.ml import feature as mlf

data.set_market('us')
with data.us_universe(index='SP500'):
    # Compute specific indicators
    indicators = mlf.ta(
        [
            'talib.RSI__real__timeperiod__14__',
            'talib.MACD__macd__fastperiod__12__slowperiod__26__signalperiod__9__',
            'talib.BBANDS__upperband__timeperiod__20__nbdevup__2.0__nbdevdn__2.0__matype__0__',
        ],
        resample='W',
    )

    # Random parameter sets for chosen indicators
    names = [
        name
        for name in mlf.ta_names(n=3)
        if name.startswith(('talib.RSI__', 'talib.MACD__', 'talib.ATR__'))
    ]
    random_indicators = mlf.ta(names, resample='W')

function

finlab.ml.ta

ta(feature_names, factories, resample, start_time, end_time, adj, cpu, **kwargs) -> pd.DataFrame

Calculate technical indicator values for a list of feature names.

Parameters

feature_names list[str] | None: A list of technical indicator feature names. Defaults to None.
factories dict[str, TalibIndicatorFactory] | None: A dictionary of factories to generate technical indicators. Defaults to {"talib": TalibIndicatorFactory()}.
resample str | None: The frequency to resample the data to. Defaults to None.
start_time str | None: The start time of the data. Defaults to None.
end_time str | None: The end time of the data. Defaults to None.
**kwargs Any default {}: Additional keyword arguments to pass to the resampler function.
adj bool default False
cpu int default -1

Returns

pd.DataFrame: pd.DataFrame: technical indicator feature names and their corresponding values.

mlf.ta_names()

Lists TA-Lib indicator names, drawing n random parameter sets for each indicator, ready for mlf.ta().

from finlab.ml import feature as mlf

# Generate all TA-Lib indicators with 10 random parameter sets each
indicator_names = mlf.ta_names(n=10)
print(f"Total {len(indicator_names)} indicators")

# View examples
for name in indicator_names[:5]:
    print(name)
# Output (parameters are drawn at random; indicators without parameters appear once):
# talib.HT_DCPERIOD__real__
# talib.HT_DCPHASE__real__
# talib.HT_PHASOR__inphase__
# talib.HT_PHASOR__quadrature__
# talib.HT_SINE__sine__

n sets how many names you get:

`n`	Meaning	Names
`1`	One random parameter set per indicator	About 130 with TA-Lib 0.8
`5`	Up to 5 random parameter sets per indicator	About 560
`10`	More variety, but longer computation time	About 1,070

Start with n=1 to test feasibility, then increase. Too many indicators mean long computation, high memory use and a higher risk of overfitting, so use feature selection to reduce the count.

function

finlab.ml.ta_names

ta_names(lb, ub, n, factory) -> list[str]

Generate a list of technical indicator feature names.

Parameters

lb int default 1: The lower bound of the multiplier of the default parameter for the technical indicators.
ub int default 10: The upper bound of the multiplier of the default parameter for the technical indicators.
n int default 1: The number of random samples for each technical indicator.
factory IndicatorFactory: A factory object to generate technical indicators. Defaults to TalibIndicatorFactory.

Returns

list[str]: list[str]: A list of technical indicator feature names.

Examples

import finlab.ml.feature as f


# method 1: generate each indicator with random parameters
features = f.ta()

# method 2: generate specific indicator
feature_names = ['talib.MACD__macdhist__fastperiod__52__slowperiod__212__signalperiod__75__']
features = f.ta(feature_names, resample='W')

# method 3: generate some indicator
feature_names = f.ta_names()
features = f.ta(feature_names)

Build training labels

A label is the target the model learns to predict. All three functions take the features' index, features.index, and return a Series aligned to the same (datetime, instrument) pairs.

mll.return_percentage()

Each sample's return over the period periods after its feature date, measured from the next trading day's price. This is the most common label.

from finlab import data
from finlab.ml import feature as mlf, label as mll

data.set_market('us')
features = mlf.combine({'rsi': data.indicator('RSI')}, resample='W')

# Generate label: predict next 1-week return
label = mll.return_percentage(features.index, resample='W', period=1)

# Generate label: predict next 4-week return
label_4w = mll.return_percentage(features.index, resample='W', period=4)

Parameter	Type	Default	Description
`index`	`pd.Index`	required	The features' (`datetime`, `instrument`) index
`resample`	`str`, `pd.DatetimeIndex` or `None`	`None`	Resample frequency or date anchors; match the features
`period`	`int`	`1`	How many periods ahead. With `resample=None` it counts trading days, even for weekly features
`trade_at_price`	`str`	`'close'`	Which price to measure with
`bfill`	`bool`	`False`	`True` fills missing prices while a symbol is listed with the next valid price (before listing and after delisting stay empty)
`**kwargs`			Passed to the resampler

Choosing period:

period=1: short-term prediction, suits weekly or monthly rebalancing
period=4: medium-term prediction, suits quarterly rebalancing
A larger period gives steadier predictions but slower signals
Match it to the strategy's rebalancing frequency

mll.daytrading_percentage()

The return from buying at the next trading day's open and selling at its close, for day-trading strategies.

from finlab import data
from finlab.ml import feature as mlf, label as mll

data.set_market('us')

# Build daily features
features = mlf.combine({'rsi': data.indicator('RSI')}, resample='D')

# Label: return from the next trading day's open to its close
label = mll.daytrading_percentage(features.index)

function

finlab.ml.daytrading_percentage

daytrading_percentage(index, resample, **kwargs) -> pd.Series

Calculate the percentage change of market prices over a given period.

Parameters

index pd.Index: A multi-level index of datetime and instrument.
resample str | None default 'D': The resample frequency for the output data. Defaults to "D".
**kwargs Any default {}: Additional arguments to be passed to the resampler function.

Returns

pd.Series: pd.Series: A pd.Series containing the percentage change of stock prices.

mll.maximum_adverse_excursion()

The largest drop over the next period periods (maximum adverse excursion), useful for training risk models.

from finlab import data
from finlab.ml import feature as mlf, label as mll

data.set_market('us')
features = mlf.combine({'rsi': data.indicator('RSI')}, resample='D')

# Compute maximum drawdown over next N periods
mae_label = mll.maximum_adverse_excursion(features.index, period=5)

# Can be used to train risk prediction models
# More negative values indicate higher risk

function

finlab.ml.maximum_adverse_excursion

maximum_adverse_excursion(index, period, trade_at_price) -> pd.Series

Calculate the maximum adverse excursion of market prices over a given period.

Parameters

index pd.Index: A multi-level index of datetime and instrument.
period int default 1: The number of periods to calculate the percentage change over. Defaults to 1.
trade_at_price str default 'close': The price for execution. Defaults to close.

Returns

pd.Series: pd.Series: A pd.Series containing the maximum adverse excursion.

Train a model and predict

finlab.ml.qlib wraps qlib models behind one interface: model.fit(X_train, y_train) trains, and model.predict(X_test) returns a FinlabDataFrame of dates by symbols.

from finlab import data
from finlab.ml import feature as mlf, label as mll
import finlab.ml.qlib as q

# 1. Prepare features
data.set_market('us')
with data.us_universe(index='SP500'):
    raw_close = data.get('price:close')  # unadjusted price for valuation
    market_value = data.get('etl:us_market_value')
    eps = data.get('us_income_statement:eps_diluted')
    equity = data.get('us_balance_sheet:total_stockholders_equity')

ttm_eps = eps.rolling(4).sum()
features = mlf.combine(
    {
        'pb': market_value / equity[equity > 0],
        'pe': raw_close / ttm_eps[ttm_eps > 0],
    },
    resample='W',
)

# 2. Prepare labels
label = mll.return_percentage(features.index, resample='W', period=1)

# 3. Split train/test sets, keeping rows where the features and label are known
complete = features.notna().all(axis=1) & label.notna()
is_train = features.index.get_level_values('datetime') < '2020-01-01'
X_train, y_train = features[complete & is_train], label[complete & is_train]
X_test = features[~is_train].dropna()

# 4. Build and train model
model = q.XGBModel()  # Options: q.LGBModel(), q.CatBoostModel(), q.LinearModel()
model.fit(X_train, y_train)

# 5. Predict
y_pred = model.predict(X_test)

Model	Description	Pros	Cons	Good for
`LGBModel()`	LightGBM	Fast, good performance, low memory usage	Requires lightgbm	Beginners: balanced speed and performance
`XGBModel()`	XGBoost	Stable, high interpretability	Slower training	Stability
`CatBoostModel()`	CatBoost	Handles categorical features well, low overfitting risk	High memory usage	Categorical features
`LinearModel()`	Linear regression	Simple, fast, low overfitting risk	Usually lower performance	Quick idea validation

To avoid overfitting, start with LinearModel() as a baseline, then try more complex models.

function

finlab.ml.LGBModel

LGBModel() -> WrapperModel

LGBModel is a wrapper model for LightGBM model.

import finlab.ml.qlib as q

# build X_train, y_train, X_test

model = q.LGBModel()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

function

finlab.ml.XGBModel

XGBModel() -> WrapperModel

XGBModel is a wrapper model for XGBoost model.

import finlab.ml.qlib as q

# build X_train, y_train, X_test

model = q.XGBModel()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

function

finlab.ml.CatBoostModel

CatBoostModel() -> WrapperModel

CatBoostModel is a wrapper model for CatBoost model.

import finlab.ml.qlib as q

# build X_train, y_train, X_test

model = q.CatBoostModel()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

function

finlab.ml.LinearModel

LinearModel() -> WrapperModel

LinearModel is a wrapper model for Linear model.

import finlab.ml.qlib as q

# build X_train, y_train, X_test

model = q.LinearModel()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

Common questions

What happens if feature and label resample frequencies don't match?

Mismatched frequencies cause date misalignment, preventing correct feature-label pairing:

# Wrong: Inconsistent resample
features = mlf.combine({...}, resample='W')  # Weekly
label = mll.return_percentage(features.index, resample='D', period=1)  # Daily
# -> Labels measure 1-day returns, not 1-week returns

# Correct: Consistent resample
features = mlf.combine({...}, resample='W')
label = mll.return_percentage(features.index, resample='W', period=1)

How do I prevent overfitting?

# Method 1: Reduce feature count
features = mlf.ta(mlf.ta_names(n=1)[:20])  # Only 20 indicators

# Method 2: Use regularization (reduce model complexity)
# LightGBM supports L1/L2 regularization, tree depth limits, etc.
# Method 3: Use a simple model
model = q.LinearModel()  # Linear model is less prone to overfitting

# Method 4: Out-of-sample testing
# Ensure test set performance is close to training set (no more than 50% gap)

Training is too slow. What can I do?

# Method 1: Reduce data range
data.truncate_start = '2020-01-01'

# Method 2: Reduce feature count
features = mlf.ta(mlf.ta_names(n=1)[:10])  # Only 10 indicators

# Method 3: Use resample='W' or 'M' (instead of 'D')
features = mlf.combine({...}, resample='W')  # Weekly data is faster

How do I handle missing values?

# First check the missing value situation
print(f"Feature missing ratio: {features.isna().sum().sum() / features.size:.2%}")
print(f"Label missing ratio: {label.isna().sum() / len(label):.2%}")

# Method 1: Drop samples with missing values
features_clean = features.dropna()
label_clean = label.dropna()

# Method 2: Forward fill each stock's own history (suitable for time series)
features_filled = features.groupby(level='instrument').ffill()
label_filled = label.groupby(level='instrument').ffill()

# Method 3: Fill with 0 (suitable for technical indicators)
features_zero = features.fillna(0)

How do I turn predictions into trading signals?

model.predict() returns a FinlabDataFrame of dates by symbols, so the usual selection methods work on it:

# Method 1: Buy top N
position = pred.is_largest(30)

# Method 2: Set threshold
position = pred > pred.quantile(0.8)  # Buy top 20%

# Method 3: Long-short strategy
long_position = pred.is_largest(20)  # Long top 20
short_position = pred.is_smallest(20)  # Short bottom 20
position = long_position.astype(float) - short_position.astype(float)

# Method 4: Allocate weights by prediction rank (weights sum to 1 each date)
scores = pred.rank(axis=1, pct=True)
position = scores.div(scores.sum(axis=1), axis=0)

Next steps

Machine learning strategy workflow The end-to-end guide, from features and labels to training and backtesting.
Machine learning strategy development Feature engineering, label design, model training and avoiding overfitting in depth.
Technical indicators Compute TA-Lib indicators across many stocks at once.
Backtesting Backtest the predicted positions with sim().

For the qlib framework, see the qlib documentation; for every indicator, see the TA-Lib indicator list.

Was this page helpful?