MLOps Inside Snowflake: Feature Store, ML Jobs, Model Registry, and Model Monitoring

Most Snowflake customers already train models somewhere. The hard part is never the fit() call — it is the plumbing around it: how features are defined once and reused, how a model artifact gets versioned and governed, where training actually runs, and how predictions get back into a table that the business can query. Snowflake ML now covers that whole loop inside the account, so the data never leaves and the governance you already built (RBAC, masking policies, lineage) keeps applying.

This tutorial walks the loop end to end: Feature Store → training on ML Jobs / Container Runtime → Model Registry → batch and real-time inference → monitoring. It assumes you know Python and Snowpark basics; if you are still moving PySpark code across, read our PySpark-to-Snowpark migration guide first.

0. What you need

CREATE DATABASE IF NOT EXISTS ml_prod;
CREATE SCHEMA IF NOT EXISTS ml_prod.features;
CREATE SCHEMA IF NOT EXISTS ml_prod.models;

CREATE WAREHOUSE IF NOT EXISTS ml_wh
  WAREHOUSE_SIZE = 'MEDIUM'
  AUTO_SUSPEND = 60
  INITIALLY_SUSPENDED = TRUE;

Python side:

pip install "snowflake-ml-python>=1.8" snowflake-snowpark-python

A word on roles before you start: create a dedicated ML_ENGINEER functional role that holds USAGE on ml_prod, CREATE MODEL on ml_prod.models, and read access to the source tables. Models and feature views are schema-level objects — they inherit the same grant model as tables, which is exactly why doing this inside Snowflake is cheaper to audit than a side-car ML platform.

1. Feature Store: define features once

The Feature Store is a thin, opinionated layer over Snowflake objects. An entity is the join key (a customer, an account, a device). A feature view is a query that produces features for that entity, registered with a refresh frequency so Snowflake materialises it as a Dynamic Table under the hood.

from snowflake.snowpark import Session
from snowflake.ml.feature_store import FeatureStore, Entity, FeatureView, CreationMode

session = Session.builder.getOrCreate()

fs = FeatureStore(
    session=session,
    database="ML_PROD",
    name="FEATURES",
    default_warehouse="ML_WH",
    creation_mode=CreationMode.CREATE_IF_NOT_EXIST,
)

customer = Entity(name="CUSTOMER", join_keys=["CUSTOMER_ID"],
                  desc="Billing customer")
fs.register_entity(customer)

df = session.sql("""
    SELECT
        o.customer_id                                   AS customer_id,
        COUNT(*)                                        AS orders_90d,
        SUM(o.amount)                                   AS spend_90d,
        DATEDIFF('day', MAX(o.order_ts), CURRENT_DATE()) AS days_since_last_order,
        AVG(o.amount)                                   AS avg_order_value
    FROM analytics.public.orders o
    WHERE o.order_ts >= DATEADD('day', -90, CURRENT_DATE())
    GROUP BY 1
""")

fv = FeatureView(
    name="CUSTOMER_ACTIVITY",
    entities=[customer],
    feature_df=df,
    refresh_freq="1 hour",       # becomes a Dynamic Table
    desc="Rolling 90-day order behaviour",
)
fv = fs.register_feature_view(feature_view=fv, version="V1")

Two things matter here and both are about discipline, not code:

• refresh_freq is a cost lever. Every feature view with a refresh frequency is a Dynamic Table that runs on a warehouse forever. Ten "1 minute" feature views on a LARGE warehouse will dominate your ML bill. Start hourly; tighten only where the model actually degrades.
• Versions are immutable. V1 of a feature view is frozen. Changing the SQL means registering V2 — which is precisely how you avoid the classic silent skew where someone redefines spend_90d and last month's model quietly starts scoring garbage.

Generating a training set without leakage

The point of a feature store is the point-in-time join. Give it a spine of entity IDs plus the label timestamp, and it retrieves feature values as of that timestamp instead of today's values.

spine = session.sql("""
    SELECT customer_id, label_ts, churned AS label
    FROM ml_prod.features.churn_labels
""")

train_ds = fs.generate_dataset(
    name="CHURN_TRAIN",
    version="2026_01",
    spine_df=spine,
    features=[fv],
    spine_timestamp_col="LABEL_TS",
    spine_label_cols=["LABEL"],
)

A Dataset is a first-class, versioned, immutable Snowflake object. When a regulator or a customer asks what data trained the model that made a decision, the answer is a name and a version, not a Slack thread.

2. Train where the data is

You have three escalating options. Pick the cheapest one that fits.

(a) In a warehouse, via Snowpark ML modelling APIs. Good for scikit-learn/XGBoost-shaped problems on data that fits distributed CPU training.

from snowflake.ml.modeling.xgboost import XGBClassifier

FEATURES = ["ORDERS_90D", "SPEND_90D", "DAYS_SINCE_LAST_ORDER", "AVG_ORDER_VALUE"]

model = XGBClassifier(
    input_cols=FEATURES,
    label_cols=["LABEL"],
    output_cols=["PREDICTION"],
    max_depth=6,
    n_estimators=300,
)
model.fit(train_ds.read.to_snowpark_dataframe())

(b) On Container Runtime with ML Jobs. When you want plain open-source code (import xgboost, import lightgbm, PyTorch), GPUs, or a pip environment you control, submit the function as a job that runs on a compute pool. Your laptop stops being production.

from snowflake.ml.jobs import remote

@remote(
    compute_pool="ML_GPU_POOL",
    stage_name="@ml_prod.models.job_stage",
    pip_requirements=["xgboost==2.1.1", "scikit-learn==1.5.2"],
)
def train_churn(session, dataset_name: str, version: str):
    import xgboost as xgb
    from snowflake.ml.dataset import load_dataset

    pdf = load_dataset(session, dataset_name, version).read.to_pandas()
    X = pdf[["ORDERS_90D", "SPEND_90D", "DAYS_SINCE_LAST_ORDER", "AVG_ORDER_VALUE"]]
    y = pdf["LABEL"]
    booster = xgb.XGBClassifier(max_depth=6, n_estimators=300).fit(X, y)
    return booster

job = train_churn("ML_PROD.FEATURES.CHURN_TRAIN", "2026_01")
print(job.status)          # PENDING -> RUNNING -> DONE
print(job.get_logs())
booster = job.result()

Compute pools bill per node-second while they are up, independent of your warehouses. Set AUTO_SUSPEND_SECS on the pool and use MIN_NODES = 0 for training pools — an idle GPU pool is the single most expensive mistake in Snowflake ML.

(c) Cortex AI functions instead of training at all. For text classification, sentiment, extraction, translation and summarisation, AI_CLASSIFY, AI_COMPLETE and friends often beat a bespoke model on total cost of ownership. Train a model when you have a genuine tabular prediction problem; call a Cortex function when you have a language problem.

3. Model Registry: version the artifact

from snowflake.ml.registry import Registry

reg = Registry(session=session, database_name="ML_PROD", schema_name="MODELS")

mv = reg.log_model(
    booster,
    model_name="CHURN_CLASSIFIER",
    version_name="V3",
    sample_input_data=X.head(100),          # infers the signature
    conda_dependencies=["xgboost==2.1.1"],
    comment="Adds avg_order_value; trained on CHURN_TRAIN 2026_01",
    metrics={"auc": 0.871, "pr_auc": 0.44},
)

reg.get_model("CHURN_CLASSIFIER").default = "V3"

The registry object is a schema-level MODEL, so it shows up in SHOW MODELS, obeys grants, and appears in lineage. Two habits worth enforcing in review:

• Always pass sample_input_data or an explicit signature. A model without a signature will accept a column order change silently.
• Always record metrics and a comment naming the training dataset version. "Which dataset trained V3?" should be answerable with SQL:

SHOW VERSIONS IN MODEL ml_prod.models.churn_classifier;
SELECT * FROM TABLE(ml_prod.models.churn_classifier!SHOW_METRICS());

Promotion between environments is a CREATE MODEL ... CLONE or a share, not a re-train. If dev and prod are separate accounts, replicate the models schema — the artifact that passed validation is the artifact that serves traffic.

4. Inference: batch, SQL, and real-time

Batch from Python:

preds = mv.run(scoring_df, function_name="predict_proba")
preds.write.mode("overwrite").save_as_table("ML_PROD.MODELS.CHURN_SCORES")

From SQL — which is what makes this stick, because your analysts and your dbt models can use it without touching Python:

SELECT
    c.customer_id,
    ml_prod.models.churn_classifier!PREDICT_PROBA(
        c.orders_90d, c.spend_90d, c.days_since_last_order, c.avg_order_value
    ):"1"::FLOAT AS churn_probability
FROM ml_prod.features.customer_activity c;

Wrap that in a Dynamic Table and scoring becomes a declarative pipeline with a lag target rather than a cron job someone has to babysit.

Real-time: create a service from the model version on a compute pool and you get an HTTP endpoint with the same artifact behind it.

ALTER MODEL ml_prod.models.churn_classifier
  CREATE SERVICE churn_svc
  VERSION = 'V3'
  COMPUTE_POOL = ml_inference_pool
  MIN_INSTANCES = 1
  MAX_INSTANCES = 4
  INGRESS_ENABLED = TRUE;

Only do this when you genuinely need sub-second, per-request scoring. A Dynamic Table refreshing every five minutes serves the majority of "real-time" requirements at a fraction of the cost, because a warehouse suspends and a compute pool instance does not.

5. Feature retrieval at inference time

Do not rewrite the feature SQL in your scoring job. Retrieve from the same feature views you trained on:

scoring_spine = session.table("ML_PROD.FEATURES.CUSTOMERS_TO_SCORE")
scoring_df = fs.retrieve_feature_values(
    spine_df=scoring_spine,
    features=[fv],
)

Training/serving skew almost always enters through a duplicated transformation, not through a bad model. One definition, two consumers.

6. Monitoring

Register a model monitor and Snowflake tracks drift and, once labels arrive, performance:

CREATE MODEL MONITOR ml_prod.models.churn_monitor
  WITH
    MODEL = ml_prod.models.churn_classifier
    VERSION = 'V3'
    FUNCTION = 'predict_proba'
    SOURCE = ml_prod.models.churn_scores
    BASELINE = ml_prod.models.churn_training_snapshot
    TIMESTAMP_COLUMN = scored_at
    PREDICTION_SCORE_COLUMNS = (churn_probability)
    ACTUAL_CLASS_COLUMNS = (churned)
    ID_COLUMNS = (customer_id)
    WAREHOUSE = ml_wh
    REFRESH_INTERVAL = '1 day'
    AGGREGATION_WINDOW = '1 day';

Then pair it with an alert so drift reaches a human:

CREATE OR REPLACE ALERT ml_prod.models.churn_drift_alert
  WAREHOUSE = ml_wh
  SCHEDULE = 'USING CRON 0 7 * * * UTC'
  IF (EXISTS (
      SELECT 1
      FROM TABLE(MODEL_MONITOR_DRIFT_METRIC(
          'ML_PROD.MODELS.CHURN_MONITOR', 'DIFFERENCE_OF_MEANS',
          'CHURN_PROBABILITY', '1 day',
          DATEADD('day', -7, CURRENT_DATE()), CURRENT_DATE()))
      WHERE ABS(metric_value) > 0.05
  ))
  THEN CALL SYSTEM$SEND_EMAIL('ml_alerts', 'data-eng@acme.com',
        'Churn model drift', 'Prediction mean shifted >0.05 in the last 7 days.');

If you already stood up event tables and alerts from our Snowflake Trail tutorial, route these through the same notification integration rather than inventing a second paging path.

A migration path that works

Teams with an existing SageMaker/Vertex/Databricks ML footprint rarely benefit from a big-bang move. The order that consistently works:

1. Move feature engineering first. It is already SQL, it is the biggest source of skew, and the Feature Store pays for itself immediately.
2. Move batch inference second. Bring the artifact in with log_model and serve it from SQL; you delete an export pipeline and a copy of the data.
3. Move training third, onto ML Jobs, once the pip environments are pinned.
4. Move real-time serving last, or never. Endpoints are the least differentiated part of the stack.

Cost and governance checklist

• Feature view refresh_freq reviewed quarterly; anything under 15 minutes needs a justification.
• Training compute pools: MIN_NODES = 0, aggressive AUTO_SUSPEND_SECS, and a budget on the pool.
• ML_ENGINEER role separate from ANALYST; CREATE MODEL granted only in ml_prod.models.
• Masking policies verified against training sets — a feature view can absolutely leak a column an analyst is masked out of, because the feature view runs as its owner.
• Every model version carries metrics, a comment, and the dataset version that produced it.
• A monitor plus an alert for every model that touches a customer-facing decision.

Where teams get stuck

The three failure modes we are most often called in to fix: feature logic duplicated between a dbt model and a scoring script (fix the definition, not the model); compute pools left running between quarterly retrains (six figures, occasionally); and models logged without signatures, so an upstream column rename produces confidently wrong predictions rather than an error.

If you are standing up Snowflake ML and want the feature store, registry and monitoring patterns laid down correctly before the first model ships — or you have models living outside Snowflake and want a staged migration plan — get in touch. Our Snowflake architects do this work as short, scoped engagements.