Skip to content

Commit

Permalink
doc updates (microsoft#3415)
Browse files Browse the repository at this point in the history
  • Loading branch information
goodship1 authored May 1, 2023
1 parent 7ddc3b0 commit b4b63f5
Showing 1 changed file with 15 additions and 15 deletions.
30 changes: 15 additions & 15 deletions docs/_tutorials/cifar-10.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,21 +8,21 @@ If you haven't already, we advise you to first read through the
[Getting Started](/getting-started/) guide before stepping through this
tutorial.

In this tutorial we will be adding DeepSpeed to CIFAR-10 model, which is small image classification model.
In this tutorial we will be adding DeepSpeed to the CIFAR-10 model, which is a small image classification model.

First we will go over how to run original CIFAR-10. Then we will proceed step-by-step in enabling this model to run with DeepSpeed.
First we will go over how to run the original CIFAR-10 model. Then we will proceed step-by-step in enabling this model to run with DeepSpeed.



## Running Original CIFAR-10

Original model code from [CIFAR-10 Tutorial](https://github.com/pytorch/tutorials/blob/main/beginner_source/blitz/cifar10_tutorial.py), We've copied this repo under [DeepSpeedExamples/training/cifar/](https://github.com/microsoft/DeepSpeedExamples/tree/master/training/cifar) and made it available as a submodule. To download, execute:
Original model code from the [CIFAR-10 Tutorial](https://github.com/pytorch/tutorials/blob/main/beginner_source/blitz/cifar10_tutorial.py), We've copied this repo under [DeepSpeedExamples/training/cifar/](https://github.com/microsoft/DeepSpeedExamples/tree/master/training/cifar) and made it available as a submodule. To download, execute:

```bash
git submodule update --init --recursive
```

To install requirements for CIFAR-10:
To install the requirements for the CIFAR-10 model:
```bash
cd DeepSpeedExamples/cifar
pip install -r requirements.txt
Expand Down Expand Up @@ -82,22 +82,22 @@ The first step to apply DeepSpeed is adding DeepSpeed arguments to CIFAR-10 mode

parser=argparse.ArgumentParser(description='CIFAR')

#data
# cuda
# Data.
# Cuda.
parser.add_argument('--with_cuda', default=False, action='store_true',
help='use CPU in case there\'s no GPU support')
parser.add_argument('--use_ema', default=False, action='store_true',
help='whether use exponential moving average')

# train
# Train.
parser.add_argument('-b', '--batch_size', default=32, type=int,
help='mini-batch size (default: 32)')
parser.add_argument('-e', '--epochs', default=30, type=int,
help='number of total epochs (default: 30)')
parser.add_argument('--local_rank', type=int, default=-1,
help='local rank passed from distributed launcher')

# Include DeepSpeed configuration arguments
# Include DeepSpeed configuration arguments.
parser = deepspeed.add_config_arguments(parser)

args=parser.parse_args()
Expand All @@ -123,16 +123,16 @@ def initialize(args,
collate_fn=None):
```

Here we initialize DeepSpeed with CIFAR-10 model (`net`), `args`, `parameters` and `trainset`:
Here we initialize DeepSpeed with the CIFAR-10 model (`net`), `args`, `parameters` and `trainset`:

```python
parameters = filter(lambda p: p.requires_grad, net.parameters())
args=add_argument()

# Initialize DeepSpeed to use the following features
# 1) Distributed model
# 2) Distributed data loader
# 3) DeepSpeed optimizer
# 1) Distributed model.
# 2) Distributed data loader.
# 3) DeepSpeed optimizer.
model_engine, optimizer, trainloader, _ = deepspeed.initialize(args=args, model=net, model_parameters=parameters, training_data=trainset)

```
Expand All @@ -155,7 +155,7 @@ The `model` returned by `deepspeed.initialize` is the _DeepSpeed Model Engine_ t

```python
for i, data in enumerate(trainloader):
# get the inputs; data is a list of [inputs, labels]
# Get the inputs; data is a list of [inputs, labels].
inputs = data[0].to(model_engine.device)
labels = data[1].to(model_engine.device)

Expand Down Expand Up @@ -206,13 +206,13 @@ The next step to use DeepSpeed is to create a configuration JSON file (ds_config

### Run CIFAR-10 Model with DeepSpeed Enabled

To start training CIFAR-10 model with DeepSpeed applied, execute the following command, it will use all detected GPUs by default.
To start training the CIFAR-10 model with DeepSpeed applied, execute the following command, it will use all detected GPUs by default.

```bash
deepspeed cifar10_deepspeed.py --deepspeed_config ds_config.json
```

DeepSpeed usually prints more training details for user to monitor, including training settings, performance statistics and loss trends.
DeepSpeed usually prints more training details for the user to monitor, including training settings, performance statistics and loss trends.
```
deepspeed.pt cifar10_deepspeed.py --deepspeed_config ds_config.json
Warning: Permanently added '[192.168.0.22]:42227' (ECDSA) to the list of known hosts.
Expand Down

0 comments on commit b4b63f5

Please sign in to comment.