Numerai — Complete Guide

What Is Numerai?

A hedge fund that runs on crowd-sourced ML models. Every week, thousands of data scientists submit predictions on obfuscated financial data. Numerai uses the best predictions to trade real money. You stake NMR (their crypto token) on your model and earn more if you're right — or lose if you're wrong.

Why It's Unique

Your model runs in PRODUCTION in a real hedge fund
Weekly income possible (not one-time prize)
Data is fully obfuscated: no finance knowledge needed
100% free to start (stake only when confident)

URL

Tournament: numer.ai
Docs: docs.numer.ai
Signals (alternative): signals.numer.ai
Forum: forum.numer.ai ← Read this weekly

How It Works

Download the dataset (free, weekly) — ~5M rows, ~2,400 features
Train a model on obfuscated features (feature_0, feature_1, etc.)
Submit predictions for the live round
After 20 days of scoring, models with positive scores earn NMR
Models with negative scores have NMR burned (you lose your stake)

Getting Started (No Staking First)

# pip install numerapi
import numerapi
napi = numerapi.NumerAPI()

# Download training data
napi.download_dataset("v5.0/train.parquet", "train.parquet")
napi.download_dataset("v5.0/validation.parquet", "validation.parquet")
napi.download_dataset("v5.0/live.parquet", "live.parquet")  # this week's live data

Key Technical Concepts

Concept	What It Is	Why It Matters
Era	One time period (a "week" of stock data)	Never random-split across eras — data is temporally correlated
Feature neutralization	Remove market-correlated features from predictions	Reduces systemic risk; Numerai rewards unique signals
Era boosting	Upweight hard eras during training	Prevents overfitting to easy periods
CORR	Main scoring metric (Spearman correlation with targets)	Your primary signal quality measure
MMC	Meta Model Contribution — bonus for unique predictions	Where the real edge is; target this
FNC	Feature Neutral Correlation — CORR after neutralization	More stable than raw CORR
TC	True Contribution to Numerai's portfolio	Advanced metric used in payouts

Era-Aware Cross-Validation (Most Important)

Never use random k-fold. Eras are correlated across time — random splits cause massive data leakage.

import pandas as pd
import numpy as np
from lightgbm import LGBMRegressor

# Load data
train = pd.read_parquet("train.parquet")

# Identify feature columns and target
feature_cols = [c for c in train.columns if c.startswith("feature_")]
target_col = "target"
era_col = "era"

# Era-aware cross-validation: hold out entire eras as test sets
def era_cv_split(df, n_splits=5):
    """Split by era groups, not individual rows."""
    eras = df[era_col].unique()
    era_groups = np.array_split(eras, n_splits)
    for i, test_eras in enumerate(era_groups):
        train_eras = np.concatenate([era_groups[j] for j in range(n_splits) if j != i])
        train_idx = df[df[era_col].isin(train_eras)].index
        test_idx = df[df[era_col].isin(test_eras)].index
        yield train_idx, test_idx

oof_preds = np.zeros(len(train))
models = []

for fold, (tr_idx, val_idx) in enumerate(era_cv_split(train, n_splits=5)):
    X_tr = train.loc[tr_idx, feature_cols]
    y_tr = train.loc[tr_idx, target_col]
    X_val = train.loc[val_idx, feature_cols]
    y_val = train.loc[val_idx, target_col]
    
    model = LGBMRegressor(
        n_estimators=2000,
        learning_rate=0.01,
        num_leaves=31,
        colsample_bytree=0.1,  # feature subsampling — crucial for Numerai
        subsample=0.5,
        n_jobs=-1
    )
    model.fit(X_tr, y_tr, eval_set=[(X_val, y_val)], callbacks=[lgb.early_stopping(100)])
    oof_preds[val_idx] = model.predict(X_val)
    models.append(model)
    
    # Score each fold (Spearman correlation per era)
    fold_corr = train.loc[val_idx].groupby(era_col).apply(
        lambda g: g[target_col].corr(pd.Series(oof_preds[g.index], index=g.index), method='spearman')
    ).mean()
    print(f"Fold {fold+1} mean CORR: {fold_corr:.4f}")

Era Boosting — Upweight Hard Eras

def compute_era_weights(df, predictions, alpha=0.5):
    """
    Upweight eras where the model performed worst.
    alpha: how strongly to upweight (0 = uniform, 1 = max upweighting)
    """
    era_scores = df.groupby(era_col).apply(
        lambda g: g[target_col].corr(
            pd.Series(predictions[g.index], index=g.index), method='spearman'
        )
    )
    # Invert and normalize: worst eras get highest weight
    era_weights = 1 / (era_scores.rank() + 1)
    era_weights = alpha * era_weights / era_weights.max() + (1 - alpha)
    return df[era_col].map(era_weights).values

Feature Neutralization — Remove Correlated Signal

After making predictions, neutralize against the "example predictions" (Numerai's own model) to ensure your signal adds unique information.

def neutralize(predictions: pd.Series, factors: pd.DataFrame, proportion=1.0):
    """
    Remove the linear component of 'factors' from 'predictions'.
    Use proportion < 1.0 to partially neutralize.
    """
    from numpy.linalg import lstsq
    
    predictions = predictions.values.reshape(-1, 1)
    factors = factors.values
    
    # Fit linear regression: predictions ~ factors
    loadings, _, _, _ = lstsq(factors, predictions, rcond=None)
    
    # Residual = predictions - factors @ loadings (neutralized)
    neutralized = predictions - proportion * (factors @ loadings)
    
    return pd.Series(neutralized.flatten())

# Usage:
example_preds = pd.read_parquet("v5.0/validation_example_preds.parquet")
neutralized_preds = neutralize(
    predictions=your_predictions,
    factors=example_preds[['prediction']],
    proportion=0.5  # 50% neutralization — tune this
)

MMC — Targeting Unique Predictions (The Real Alpha)

MMC rewards predictions that add value BEYOND what Numerai's meta-model already captures. High MMC is worth more than high CORR.

Rule of thumb: If your model is very similar to publicly shared Numerai models, your MMC will be low even if CORR is high.

To maximize MMC:

Use unusual feature subsets (not all 2,400 features)
Try fundamentally different model architectures (MLP, LightGBM, CatBoost, ensembles)
Train on different target variants (there are 20+ targets in v5 data)
Use non-linear feature combinations (interactions, embeddings)

# Train on different targets and blend
TARGETS = ['target_cyrus_v4_20', 'target_victor_v4_20', 'target_ralph_v4_20']

target_preds = []
for tgt in TARGETS:
    model = LGBMRegressor(n_estimators=1000, learning_rate=0.01, num_leaves=31)
    model.fit(X_train, train[tgt])
    target_preds.append(model.predict(X_live))

# Simple blend — diversifies your signal across targets
blended_pred = np.mean(target_preds, axis=0)

Submission

import numerapi

napi = numerapi.NumerAPI()

# Generate live predictions
live = pd.read_parquet("live.parquet")
X_live = live[feature_cols]

# Average all fold models for robustness
live_preds = np.mean([m.predict(X_live) for m in models], axis=0)

# Rank transform (Numerai prefers rank-normalized predictions)
live_preds_ranked = pd.Series(live_preds).rank(pct=True)

# Save and upload
submission = pd.DataFrame({
    'id': live.index,
    'prediction': live_preds_ranked
})
submission.to_csv("submission.csv", index=False)

# Upload to your model slot
napi.upload_predictions("submission.csv", model_id="your-model-id")

Feature Selection for Numerai

With 2,400 features, selection is crucial:

# Method 1: Use Numerai's feature metadata
feature_metadata = pd.read_json("features.json")
# Select "small" feature set (Numerai provides curated subsets)
small_features = feature_metadata[feature_metadata['feature_set'] == 'small'].index.tolist()

# Method 2: Select features with highest feature-era correlation stability
# (features that work consistently across eras, not just on average)
def feature_era_corr(df, feature, target):
    return df.groupby(era_col).apply(
        lambda g: g[feature].corr(g[target], method='spearman')
    )

era_corrs = {f: feature_era_corr(train, f, target_col) for f in feature_cols[:100]}
# Keep features with consistently positive (not just high mean) era correlation
stable_features = [f for f, corrs in era_corrs.items() if (corrs > 0).mean() > 0.55]

Winning Tips (Summary)

Era-aware CV is non-negotiable — random split gives falsely optimistic scores
Use colsample_bytree=0.1 in LightGBM — Numerai features are noisy; aggressive feature subsampling helps
Target MMC, not just CORR — unique predictions earn more via the payout multiplier
Feature neutralize to reduce correlated risk and improve FNC score
Era boost to improve out-of-era generalization
Read the forum (forum.numer.ai) — the community openly shares what's working
Run multiple model variants — blend 5–10 diverse models for the most stable score
Don't stake until you have 3+ months of positive live scores

Realistic Expectations

Not a get-rich-quick scheme. Consistent 0.02–0.05 CORR = good model
Top performers earn $500–$5,000/month from staking (at reasonable stake sizes)
NMR price fluctuates (it's crypto) — factor this into risk
Treat it as recurring passive income + real-world ML practice

Tools and Libraries

Tool	Use
`numerapi`	Official Python API for downloading data and uploading predictions
`lightgbm`	Primary model — fast, strong, handles feature noise well
`catboost`	Good alternative; handles missing values natively
`scipy.stats.spearmanr`	Manual CORR computation
`mlfinlab`	Purged CV implementation for advanced splitting

Resources

Resource	URL
Official docs	docs.numer.ai
Community forum	forum.numer.ai
Numerai GitHub	github.com/numerai
"Supermastery" notebook (public starter)	kaggle.com/code numerai tag
NumerBay (buy/sell signals)	numerbay.ai

Numerai — Complete Guide

What Is Numerai?

Why It's Unique

Your model runs in PRODUCTION in a real hedge fund
Weekly income possible (not one-time prize)
Data is fully obfuscated: no finance knowledge needed
100% free to start (stake only when confident)

URL

Tournament: numer.ai
Docs: docs.numer.ai
Signals (alternative): signals.numer.ai
Forum: forum.numer.ai ← Read this weekly

How It Works

Download the dataset (free, weekly) — ~5M rows, ~2,400 features
Train a model on obfuscated features (feature_0, feature_1, etc.)
Submit predictions for the live round
After 20 days of scoring, models with positive scores earn NMR
Models with negative scores have NMR burned (you lose your stake)

Getting Started (No Staking First)

# pip install numerapi
import numerapi
napi = numerapi.NumerAPI()

# Download training data
napi.download_dataset("v5.0/train.parquet", "train.parquet")
napi.download_dataset("v5.0/validation.parquet", "validation.parquet")
napi.download_dataset("v5.0/live.parquet", "live.parquet")  # this week's live data

Key Technical Concepts

Concept	What It Is	Why It Matters
Era	One time period (a "week" of stock data)	Never random-split across eras — data is temporally correlated
Feature neutralization	Remove market-correlated features from predictions	Reduces systemic risk; Numerai rewards unique signals
Era boosting	Upweight hard eras during training	Prevents overfitting to easy periods
CORR	Main scoring metric (Spearman correlation with targets)	Your primary signal quality measure
MMC	Meta Model Contribution — bonus for unique predictions	Where the real edge is; target this
FNC	Feature Neutral Correlation — CORR after neutralization	More stable than raw CORR
TC	True Contribution to Numerai's portfolio	Advanced metric used in payouts

Era-Aware Cross-Validation (Most Important)

Never use random k-fold. Eras are correlated across time — random splits cause massive data leakage.

import pandas as pd
import numpy as np
from lightgbm import LGBMRegressor

# Load data
train = pd.read_parquet("train.parquet")

# Identify feature columns and target
feature_cols = [c for c in train.columns if c.startswith("feature_")]
target_col = "target"
era_col = "era"

# Era-aware cross-validation: hold out entire eras as test sets
def era_cv_split(df, n_splits=5):
    """Split by era groups, not individual rows."""
    eras = df[era_col].unique()
    era_groups = np.array_split(eras, n_splits)
    for i, test_eras in enumerate(era_groups):
        train_eras = np.concatenate([era_groups[j] for j in range(n_splits) if j != i])
        train_idx = df[df[era_col].isin(train_eras)].index
        test_idx = df[df[era_col].isin(test_eras)].index
        yield train_idx, test_idx

oof_preds = np.zeros(len(train))
models = []

for fold, (tr_idx, val_idx) in enumerate(era_cv_split(train, n_splits=5)):
    X_tr = train.loc[tr_idx, feature_cols]
    y_tr = train.loc[tr_idx, target_col]
    X_val = train.loc[val_idx, feature_cols]
    y_val = train.loc[val_idx, target_col]
    
    model = LGBMRegressor(
        n_estimators=2000,
        learning_rate=0.01,
        num_leaves=31,
        colsample_bytree=0.1,  # feature subsampling — crucial for Numerai
        subsample=0.5,
        n_jobs=-1
    )
    model.fit(X_tr, y_tr, eval_set=[(X_val, y_val)], callbacks=[lgb.early_stopping(100)])
    oof_preds[val_idx] = model.predict(X_val)
    models.append(model)
    
    # Score each fold (Spearman correlation per era)
    fold_corr = train.loc[val_idx].groupby(era_col).apply(
        lambda g: g[target_col].corr(pd.Series(oof_preds[g.index], index=g.index), method='spearman')
    ).mean()
    print(f"Fold {fold+1} mean CORR: {fold_corr:.4f}")

Era Boosting — Upweight Hard Eras

def compute_era_weights(df, predictions, alpha=0.5):
    """
    Upweight eras where the model performed worst.
    alpha: how strongly to upweight (0 = uniform, 1 = max upweighting)
    """
    era_scores = df.groupby(era_col).apply(
        lambda g: g[target_col].corr(
            pd.Series(predictions[g.index], index=g.index), method='spearman'
        )
    )
    # Invert and normalize: worst eras get highest weight
    era_weights = 1 / (era_scores.rank() + 1)
    era_weights = alpha * era_weights / era_weights.max() + (1 - alpha)
    return df[era_col].map(era_weights).values

Feature Neutralization — Remove Correlated Signal

After making predictions, neutralize against the "example predictions" (Numerai's own model) to ensure your signal adds unique information.

def neutralize(predictions: pd.Series, factors: pd.DataFrame, proportion=1.0):
    """
    Remove the linear component of 'factors' from 'predictions'.
    Use proportion < 1.0 to partially neutralize.
    """
    from numpy.linalg import lstsq
    
    predictions = predictions.values.reshape(-1, 1)
    factors = factors.values
    
    # Fit linear regression: predictions ~ factors
    loadings, _, _, _ = lstsq(factors, predictions, rcond=None)
    
    # Residual = predictions - factors @ loadings (neutralized)
    neutralized = predictions - proportion * (factors @ loadings)
    
    return pd.Series(neutralized.flatten())

# Usage:
example_preds = pd.read_parquet("v5.0/validation_example_preds.parquet")
neutralized_preds = neutralize(
    predictions=your_predictions,
    factors=example_preds[['prediction']],
    proportion=0.5  # 50% neutralization — tune this
)

MMC — Targeting Unique Predictions (The Real Alpha)

MMC rewards predictions that add value BEYOND what Numerai's meta-model already captures. High MMC is worth more than high CORR.

Rule of thumb: If your model is very similar to publicly shared Numerai models, your MMC will be low even if CORR is high.

To maximize MMC:

Use unusual feature subsets (not all 2,400 features)
Try fundamentally different model architectures (MLP, LightGBM, CatBoost, ensembles)
Train on different target variants (there are 20+ targets in v5 data)
Use non-linear feature combinations (interactions, embeddings)

# Train on different targets and blend
TARGETS = ['target_cyrus_v4_20', 'target_victor_v4_20', 'target_ralph_v4_20']

target_preds = []
for tgt in TARGETS:
    model = LGBMRegressor(n_estimators=1000, learning_rate=0.01, num_leaves=31)
    model.fit(X_train, train[tgt])
    target_preds.append(model.predict(X_live))

# Simple blend — diversifies your signal across targets
blended_pred = np.mean(target_preds, axis=0)

Submission

import numerapi

napi = numerapi.NumerAPI()

# Generate live predictions
live = pd.read_parquet("live.parquet")
X_live = live[feature_cols]

# Average all fold models for robustness
live_preds = np.mean([m.predict(X_live) for m in models], axis=0)

# Rank transform (Numerai prefers rank-normalized predictions)
live_preds_ranked = pd.Series(live_preds).rank(pct=True)

# Save and upload
submission = pd.DataFrame({
    'id': live.index,
    'prediction': live_preds_ranked
})
submission.to_csv("submission.csv", index=False)

# Upload to your model slot
napi.upload_predictions("submission.csv", model_id="your-model-id")

Feature Selection for Numerai

With 2,400 features, selection is crucial:

# Method 1: Use Numerai's feature metadata
feature_metadata = pd.read_json("features.json")
# Select "small" feature set (Numerai provides curated subsets)
small_features = feature_metadata[feature_metadata['feature_set'] == 'small'].index.tolist()

# Method 2: Select features with highest feature-era correlation stability
# (features that work consistently across eras, not just on average)
def feature_era_corr(df, feature, target):
    return df.groupby(era_col).apply(
        lambda g: g[feature].corr(g[target], method='spearman')
    )

era_corrs = {f: feature_era_corr(train, f, target_col) for f in feature_cols[:100]}
# Keep features with consistently positive (not just high mean) era correlation
stable_features = [f for f, corrs in era_corrs.items() if (corrs > 0).mean() > 0.55]

Winning Tips (Summary)

Era-aware CV is non-negotiable — random split gives falsely optimistic scores
Use colsample_bytree=0.1 in LightGBM — Numerai features are noisy; aggressive feature subsampling helps
Target MMC, not just CORR — unique predictions earn more via the payout multiplier
Feature neutralize to reduce correlated risk and improve FNC score
Era boost to improve out-of-era generalization
Read the forum (forum.numer.ai) — the community openly shares what's working
Run multiple model variants — blend 5–10 diverse models for the most stable score
Don't stake until you have 3+ months of positive live scores

Realistic Expectations

Not a get-rich-quick scheme. Consistent 0.02–0.05 CORR = good model
Top performers earn $500–$5,000/month from staking (at reasonable stake sizes)
NMR price fluctuates (it's crypto) — factor this into risk
Treat it as recurring passive income + real-world ML practice

Tools and Libraries

Tool	Use
`numerapi`	Official Python API for downloading data and uploading predictions
`lightgbm`	Primary model — fast, strong, handles feature noise well
`catboost`	Good alternative; handles missing values natively
`scipy.stats.spearmanr`	Manual CORR computation
`mlfinlab`	Purged CV implementation for advanced splitting

Resources

Resource	URL
Official docs	docs.numer.ai
Community forum	forum.numer.ai
Numerai GitHub	github.com/numerai
"Supermastery" notebook (public starter)	kaggle.com/code numerai tag
NumerBay (buy/sell signals)	numerbay.ai

Numerai

Open now

How to win on Numerai

Numerai — Complete Guide

What Is Numerai?

Why It's Unique

URL

How It Works

Getting Started (No Staking First)

Key Technical Concepts

Era-Aware Cross-Validation (Most Important)

Era Boosting — Upweight Hard Eras

Feature Neutralization — Remove Correlated Signal

MMC — Targeting Unique Predictions (The Real Alpha)

Submission

Feature Selection for Numerai

Winning Tips (Summary)

Realistic Expectations

Tools and Libraries

Resources

Numerai

Open now

How to win on Numerai

Numerai — Complete Guide

What Is Numerai?

Why It's Unique

URL

How It Works

Getting Started (No Staking First)

Key Technical Concepts

Era-Aware Cross-Validation (Most Important)

Era Boosting — Upweight Hard Eras

Feature Neutralization — Remove Correlated Signal

MMC — Targeting Unique Predictions (The Real Alpha)

Submission

Feature Selection for Numerai

Winning Tips (Summary)

Realistic Expectations

Tools and Libraries

Resources

Numerai

Open now

Add the competitions calendar to your phone

How to win on Numerai

Numerai — Complete Guide

What Is Numerai?

Why It's Unique

URL

How It Works

Getting Started (No Staking First)

Key Technical Concepts

Era-Aware Cross-Validation (Most Important)

Era Boosting — Upweight Hard Eras

Feature Neutralization — Remove Correlated Signal

MMC — Targeting Unique Predictions (The Real Alpha)

Submission

Feature Selection for Numerai

Winning Tips (Summary)

Realistic Expectations

Tools and Libraries

Resources

Numerai

Open now

Add the competitions calendar to your phone

How to win on Numerai

Numerai — Complete Guide

What Is Numerai?

Why It's Unique

URL

How It Works

Getting Started (No Staking First)

Key Technical Concepts

Era-Aware Cross-Validation (Most Important)

Era Boosting — Upweight Hard Eras

Feature Neutralization — Remove Correlated Signal

MMC — Targeting Unique Predictions (The Real Alpha)

Submission

Feature Selection for Numerai

Winning Tips (Summary)

Realistic Expectations

Tools and Libraries

Resources