HomeInstall
macOSWindowsLinuxVisual Studio Code Pre-release VersionShell CompletionIDE Plugins
Get Started
Data Pipelines
Data PipelinesMetrics, Parameters, and Plots
Experiment Management
Experiment TrackingExperimenting Using Pipelines
Example Scenarios
Versioning Data and Models
Tutorial 👩‍💻
CI/CD for Machine LearningFast and Secure Data Caching HubExperiment TrackingData Registry
Tutorial 👩‍💻
User Guide
Project Structure
dvc.yaml Files.dvc Files.dvcignore FilesConfigurationInternal Files
Data Management
Remote Storage
Amazon S3Azure Blob StorageGoogle Cloud StorageGoogle DriveAliyun OSSSSH & SFTPHDFS & WebHDFSHTTPWebDAV
Cloud VersioningDiscovering and Accessing DataExternal DataLarge Dataset OptimizationModifying Large Datasets
Pipelines
Defining PipelinesRunning PipelinesRun CacheExternal Dependencies and Outputs
Experiment Management
Running ExperimentsHydra CompositionComparing ExperimentsVisualizing PlotsSharing Experiments
How to
Run DVC on WindowsStop Tracking DataUpdate Tracked DataAdd Deps or Outs to a StageResolve Merge ConflictsShare a DVC Cache
Integrations
Ml FrameworksAmazon SageMakerDatabricksEvidentlyHugging Face
Environment VariablesTroubleshootingAnonymized Usage AnalyticsGlossaryUpgrading to DVC 3.0
Command Reference
addartifacts
artifacts get
cache
cache dircache migrate
check-ignorecheckoutcommitconfigdagdata statusdestroydiffdoctorexp
exp applyexp branchexp cleanexp diffexp listexp pullexp pushexp removeexp renameexp runexp saveexp show
fetchfreezegcget-urlgetimport-dbimport-urlimportinitinstalllist-urllistmetrics
metrics showmetrics diff
moveparams
params diff
plots
plots showplots diffplots templates
pullpushqueue
queue startqueue stopqueue statusqueue logsqueue removequeue kill
remote
remote addremote modifyremote defaultremote listremote removeremote rename
removereprorootstage
stage addstage list
statusstudio
studio loginstudio logoutstudio token
unfreezeunprotectupdateversion
Python API Reference
DVCFileSystemartifacts_show()exp_show()get_url()metrics_show()open()params_show()read()scm
Contributing
DVC Core ProjectDocs and Website
Changelog VS Code Extension DVCLive
InstallHow it WorksAPI Reference
end()log_artifact()log_image()log_metric()log_param()log_params()log_plot()log_sklearn_plot()make_dvcyaml()make_report()make_summary()monitor_system()next_step()
Ml Frameworks
Fast.aiHugging Face AccelerateHugging Face TransformersKerasLightGBMMMCVOptunaPyTorchPyTorch LightningPyTorch Lightning FabricScikit-learnTensorFlowXGBoostUltralytics YOLO v8
Environment Variables
Edit on GitHub

Pipelines

If you find yourself repeating sequence of actions to get or update the results of your project, then you may already have a pipeline. For example, a data science workflow could involve:

  1. Gathering data for training and validation
  2. Extracting useful features from the training dataset
  3. (Re)training an ML model
  4. Evaluating the results against the validation set

DVC helps you define these stages in a standard YAML format (.dvc and dvc.yaml files), making your pipelinepipeline more manageable and consistent to reproduce.

See Get Started: Data Pipelines for a hands-on introduction to this topic.

Modifying Large DatasetsDefining Pipelines

🐛 Found an issue? Let us know! Or fix it:

Edit on GitHub

❓ Have a question? Join our chat, we will help you:

Discord Chat

🤝 Data on petabyte scale? Checkout our sister project:

lakeFS Docs