Kaggle Practical Tips — GPU, Notebooks, and Competition Mechanics

These are the operational tips that separate experienced competitors from beginners. The difference between knowing the algorithm and being able to run it fast at competition time.

---

GPU/TPU Quota Management

What Kaggle Gives You (Free Tier)

• GPU P100/T4: 30 hours per week (resets each Sunday)
• TPU v3-8: 30 hours per week (separate quota)
• CPU: Unlimited
• RAM: 16GB (CPU), 16GB (GPU)
• Disk: 73GB /kaggle/working

GPU Quota Strategy

# Always check your quota BEFORE heavy runs
# Settings → Account → GPU Quota

# Save GPU hours by:
# 1. Develop and debug on CPU with small data subset
small_train = train.sample(n=1000, random_state=42)  # debug run

# 2. Use GPU only for full training runs
# 3. Batch your training: 1 full GPU run = all folds, not one fold at a time
# 4. Use efficient data loading to minimize GPU idle time

# Fast dataloading (avoid GPU waiting for CPU)
from torch.utils.data import DataLoader
loader = DataLoader(
    dataset,
    batch_size=32,
    num_workers=4,      # parallel CPU workers
    pin_memory=True,    # faster CPU→GPU transfer
    prefetch_factor=2   # prefetch 2 batches ahead
)

TPU vs GPU — When to Use Each

Task	Use
PyTorch training (most DL)	GPU
TensorFlow/JAX models	TPU
Large batch training (BERT, ViT)	TPU (faster for large batches)
LightGBM/XGBoost	CPU (these don't benefit from GPU here)
RAPIDS cuML/cuDF	GPU only

---

Kaggle Datasets API — Load External Data

# From a Kaggle notebook, add external datasets without uploading files
# Go to: Notebook → + Add Data → Search dataset name

# Access in notebook:
import os
print(os.listdir('/kaggle/input/'))  # see all attached datasets

# Common pattern: load competition data + external dataset
competition_data = pd.read_csv('/kaggle/input/competition-name/train.csv')
external_data = pd.read_parquet('/kaggle/input/your-dataset-name/data.parquet')

Adding External Libraries Not Available on Kaggle

# Install packages not in Kaggle's default environment
!pip install -q tabpfn polars catboost==1.2.5

# For large packages, use a Kaggle dataset to cache them:
# 1. Upload the package as a Kaggle dataset (pip download package_name)
# 2. Load from dataset in notebook (avoids internet download each run)

---

Submission Mechanics

Submission Limits

• Daily submission limit: typically 5–10 (varies by competition)
• Final selections: choose 2 submissions for final evaluation
• "Select for submission" deadline: usually at competition close
• Private LB shakeup: final ranking uses private test set — often different from public

Never Waste Submissions Strategy

Week 1–2 (Foundation):     Submit 1–2/day to establish baseline
Week 3–N-2 (Exploration):  Submit 2–3/day, always CV-validated first
Final week:                 Submit only changes that improve CV by >0.0001
Final day:                  Select 2: (1) best CV model, (2) best LB model

Submission File Best Practices

# Always validate before submitting
sample_sub = pd.read_csv('/kaggle/input/competition-name/sample_submission.csv')

def validate_submission(submission_df, sample_df):
    """Check submission format before uploading."""
    assert list(submission_df.columns) == list(sample_df.columns), "Column mismatch"
    assert len(submission_df) == len(sample_df), f"Row count mismatch: {len(submission_df)} vs {len(sample_df)}"
    assert not submission_df.isnull().any().any(), "Contains NaN values"
    assert (submission_df['id'] == sample_df['id']).all(), "ID mismatch"
    print(f"Submission valid: {len(submission_df)} rows, columns: {list(submission_df.columns)}")

validate_submission(my_submission, sample_sub)
my_submission.to_csv('submission.csv', index=False)

---

Kaggle Notebooks Best Practices

Notebook Speed Tricks

# 1. Use Polars instead of Pandas for large datasets (5–20x faster)
import polars as pl
df = pl.read_parquet('/kaggle/input/competition/train.parquet')
# Polars: lazy evaluation, multi-threaded, zero-copy

# 2. Use cuDF for GPU-accelerated DataFrames (same API as Pandas)
import cudf  # available on GPU notebooks
df = cudf.read_parquet('/kaggle/input/competition/train.parquet')
# Same code as pandas — just import cudf instead

# 3. Reduce memory usage
def reduce_mem_usage(df):
    for col in df.columns:
        col_type = df[col].dtype
        if col_type != object:
            c_min = df[col].min()
            c_max = df[col].max()
            if str(col_type)[:3] == 'int':
                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:
                    df[col] = df[col].astype(np.int8)
                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:
                    df[col] = df[col].astype(np.int16)
                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:
                    df[col] = df[col].astype(np.int32)
            else:
                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:
                    df[col] = df[col].astype(np.float32)  # float16 causes NaN issues — use float32
    return df

Kaggle Notebook Versioning (Critical)

Every time you click "Save & Run All", Kaggle creates a new version. Use versions strategically:

Version naming convention:
V1: baseline model (LightGBM, 5-fold, no tuning)
V2: + feature engineering batch 1
V3: + pseudo-labeling round 1  
V4: + ensemble (LightGBM + CatBoost)
V5: + hyperparameter tuning
V6: + pseudo-labeling round 2
VN: final submission candidates

Never delete old versions — you may need to trace back to what worked.

Keeping Secrets Out of Notebooks

# Use Kaggle Secrets for API keys (not hardcoded strings)
from kaggle_secrets import UserSecretsClient
secrets = UserSecretsClient()
api_key = secrets.get_secret("OPENAI_API_KEY")

---

Discussion Tab — How to Use It

The Discussion tab is a competitive advantage most beginners ignore.

What to Read Every Day

1. Top pinned posts — competition organizers post clarifications here
2. EDA notebooks — someone always finds the data quirk that unlocks performance
3. "What's your LB score?" threads — calibrate where you stand
4. "Data bug" discussions — leakage or mislabeled data is found here first

What to Post

• Share non-obvious EDA findings — builds reputation, judges sometimes notice
• Ask specific questions about the metric — this is actually answered by orgs
• Post your public baseline notebook — if it gets upvotes, you build Kaggle ranking and followers

---

Reading the Leaderboard

Public LB: scored on ~30% of test data (varies by competition)
Private LB: scored on remaining 70% — this is your ACTUAL final rank

Public LB shakeup risk factors:
- Small test set → high variance → large shakeup possible
- Target imbalance → model that exploits LB imbalance falls hard
- Overfit to public LB features → collapses on private set
- Time-based split → if public/private have different distributions

Defense: Trust your CV. Ignore public LB for final 2 weeks.

Detecting Potential Data Leakage

# Check if test features are predictable from training features
# (a sign of temporal/ID leakage)
from sklearn.ensemble import GradientBoostingClassifier

# Create "is_test" target
train_flag = pd.concat([
    train.assign(is_test=0),
    test.assign(is_test=1)
])

clf = GradientBoostingClassifier()
clf.fit(train_flag[feature_cols], train_flag['is_test'])
auc = roc_auc_score(train_flag['is_test'], clf.predict_proba(train_flag[feature_cols])[:,1])
print(f"Train/test distinguishability AUC: {auc:.3f}")
# If AUC > 0.9, the datasets are very different → distribution shift
# If AUC < 0.6, train and test look similar → good sign

---

Kaggle Ranking System

Tier	Requirement
Novice	Account created
Contributor	Profile complete, first notebook/discussion
Expert	1 bronze medal (notebooks/discussions) OR silver in competitions
Master	1 gold + 2 silver competition medals
Grandmaster	5 gold competition medals

Strategy: Start with Playground competitions (no prizes but medals count). They have simpler data and shorter timelines — faster ranking gains.

---

Key URLs

Resource	URL
Competition list	kaggle.com/competitions
Playground competitions	kaggle.com/competitions?listOption=participated (filter "Playground")
My profile / rankings	kaggle.com/your-username
Winning solutions index	farid.one/kaggle-solutions/
Public notebooks by solution	kaggle.com/code (search competition name)
GPU quota check	kaggle.com/settings (Account section)