Compute in Machine Learning

Rating & reviews (0 reviews)

Study Notes

Types of compute targets in Machine Learning:
  • Local compute
    Run the experiment on the same compute target as the code used to initiate the experiment
    Physical laptop or a virtual machine such as an Azure Machine Learning compute instance
    Geat choice during development and testing with low to moderate volumes of data.
  • Compute clusters
    For experiment workloads with high scalability requirements, you can use Azure Machine Learning compute clusters.
    This is a cost-effective way to run experiments that need to handle large volumes of data or use parallel processing to distribute the workload and reduce the time it takes to run.
  • Attached compute
    If you already use an Azure-based compute environment for data science, such as a virtual machine or an Azure Databricks cluster, you can attach it to your Azure Machine Learning workspace and use it as a compute target for certain types of workload.

Using compute targets
  • Submitted experiment.
  • Run will be queued while the COMPUTE_TARGET is started, and the specified environment created.
  • The run will be processed on the compute environment.
from azureml.core import Environment, ScriptRunConfig
from azureml.core.compute import ComputeTarget

compute_name = "MY_CLUSTER_NAME"

training_cluster = ComputeTarget(workspace=ws, name=compute_name)

training_env = Environment.get(workspace=ws, name='MY_ENVIRONMENT_NAME')

script_config = ScriptRunConfig(source_directory='my_dir',

Introduction to compute targets - Training | Microsoft Learn