Here is an example image which I rotated 90 degrees:

I tested a few programs to check how widespread support for EXIF in PNGs is by now:

• macOS Preview ✅
• macOS Pages ✅
• macOS Notes ✅
• ImageMagick ✅
• ExifTool (https://exiftool.org/TagNames/PNG.html) ✅
• GitHub Preview (as you can see 😅) ❌
• Affinity Photo ❌
• Gimp ❌
• Browsers ❌
• Pixelorama ❌
• VS Code ❌