Commits

Breadcrumbs

History for

onmaster

User selector

All users

Datepicker

All time

Commit History

Commits on Jun 19, 2023

• Fix AssertionError on macOS - need to check CUDA availability for bf16

  ho2103

  committed
  1eaceae

Commits on May 17, 2023

• Use bf16 only if supported

  apivovarov

  committed
  eb33b8b

Commits on Feb 5, 2023

• add the estimation of model flops utilization (MFU), a very commonly looked at metric that estimates the token throughput in units of A100 bfloat16 peak flops (312 TFLOPS). this gives us a sense of…

  Show description for ab0718a
  karpathy

  committed
  ab0718a

• oops optimizer now demands to know device_type

  karpathy

  committed
  5809026

Commits on Feb 4, 2023

• padding 50257 -> 50304 vocab_size, the nerest multiple of 64. the biggest deal smallest optimization i've made in recent past, about 25% faster. this is because the last layer is a major latency bo…

  Show description for 77e7e04
  karpathy

  committed
  77e7e04

• who needs a dataloader? overlap the prefetching of the next batch with GPU compute, ehiding the data loading latency entirely. this saves about 1ms lol

  karpathy

  committed
  3fd4c0c

Commits on Feb 2, 2023

• small usability tweaks to bench

  karpathy

  committed
  d01863e

Commits on Jan 27, 2023

• bunch of plumbing of bias all around. measuring bias=False to be about 6% faster

  karpathy

  committed
  e808a67

Commits on Jan 8, 2023

• copy pasting what seems to work to bench,sample as well. ty @lantiga

  karpathy

  committed
  b77c2e8

Commits on Jan 5, 2023

• shuttling the poor mans configurator aside into its own file and adding it to all of train,sample,bench. because i am leaving args in globals() so i can avoid having to prepend every single variabl…

  Show description for d562b3e
  karpathy

  committed
  d562b3e

Commits on Jan 2, 2023

• rename compile_model to compile, shroter, version 2 stragglers

  karpathy

  committed
  41184a2

Commits on Dec 30, 2022

• add torch.compile by default, shows almost 1.8X improvement in throughput nice

  karpathy

  committed
  5a725d9

Commits on Dec 29, 2022

• pull out dtype up top

  karpathy

  committed
  fa57d46

• add pytorch profiler support. not sure how to support both profiler and simple benchmarking, a bit gnarly atm hmm

  karpathy

  committed
  3000cf5

• add data loading into benchmarking as well, just for completeness

  karpathy

  committed
  b760ef1

Commits on Dec 28, 2022

• add benchmarking script v0

  karpathy

  committed
  70b5d93

Loading