Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Python version 3.12 and 3.13 #1465

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

DivingDuck
Copy link
Collaborator

Build with 3.13 is still experimental. It builds the binaries but due to compiling errors in library pyglet version 1.5.29 we miss the support for 3D visualization. Beside this Pronterface and Pronsole is fully functional.

Changes in release_windows.bat

  1. Change pyglet==1.5.27 to pyglet==1.5.29
  2. Add Python 3.12 and 3.13 as build option, remove support for Python 3.7
    Python 3.13 is still experimental as there is for now no 3D visualization available because of incompatibility issues with pyglet
  3. Add library pytest to additional libraries
  4. Add removal of gcoder_line.cp???-win_amd??.pyd files with python versions that have 3 instead of 2 digits
  5. Remove folder \build in addition as it will not removed any longer automatically in the build process
  6. Remove manual fix for pillow==9.5 as no longer needed

CleanCacheFiles.bat
This batch file will clean up all pycache folders, *.pyd files and left over gcoder_line files from previous builds. For local Windows only. It is helpful when you switch between Python versions or you have problems with incompatible library files or versions.

Changes in requirements.txt
Remove dedicated version for pillow in Windows build environment
pillow < 10.0; sys_platform == 'win32'
--> to: pillow ;sys_platform == 'win32'

Changes in buildpackage-win.yml
Add support for Python 3.12 and 3.13 and mark Python 3.13 as experimental

Various fixes for visible SyntaxError warnings of invalid escape sequence when compiling with Python 3.12 and 3.13
(in question to become in Python >3.13 a syntax error)

See https://docs.python.org/dev/whatsnew/3.6.html#deprecated-python-behavior:
A backslash-character pair that is not a valid escape sequence now generates a DeprecationWarning.
Although this will eventually become a SyntaxError, that will not be for several Python releases.
(Contributed by Emanuel Barry in bpo-27364.)

and https://docs.python.org/3/reference/lexical_analysis.html, Chapter 2.4.1.1. Escape sequences:
Changed in version 3.6: Unrecognized escape sequences produce a DeprecationWarning.
Changed in version 3.12: Unrecognized escape sequences produce a SyntaxWarning. In a future Python version they will be eventually a SyntaxError.

Solution: Change string to raw string when escape sequences are involved

\printrun\pronterface.py:2099: SyntaxWarning: invalid escape sequence '\d'
self.sdfiles.append(re.sub(" \d+$", "", line.strip().lower())) # NOQA
--> to: self.sdfiles.append(re.sub(r" \d+$", "", line.strip().lower())) # NOQA

\printrun\pronsole.py:61: SyntaxWarning: invalid escape sequence '\d'
tempreading_exp = re.compile('\\bT\d*:')
--> to: tempreading_exp = re.compile(r'\\bT\d*:')

\printrun\pronsole.py:201: SyntaxWarning: invalid escape sequence '\d'
self.lineignorepattern=re.compile("ok ?\d*$|.*busy: ?processing|.*busy: ?heating|.*Active Extruder: ?\d*$")
--> to: self.lineignorepattern = re.compile(r"ok ?\d*$|.*busy: ?processing|.*busy: ?heating|.*Active Extruder: ?\d*$")

\printrun\pronsole.py:1140: SyntaxWarning: invalid escape sequence '\d'
self.sdfiles.append(re.sub(" \d+$","",line.strip().lower()))
--> to: self.sdfiles.append(re.sub(r" \d+$","",line.strip().lower()))

\printrun\gcoder.py:28: SyntaxWarning: invalid escape sequence '('
gcode_exp = re.compile("\([^\(\)]*\)|;.*|[/\*].*\n|([%s])\s*([-+]?[0-9]*\.?[0-9]*)" % to_parse)
--> to: gcode_exp = re.compile(r"\([^\(\)]*\)|;.*|[/\*].*\n|([%s])\s*([-+]?[0-9]*\.?[0-9]*)" % to_parse)

\printrun\gcoder.py:29: SyntaxWarning: invalid escape sequence '('
gcode_strip_comment_exp = re.compile("\([^\(\)]*\)|;.*|[/\*].*\n")
--> to: gcode_strip_comment_exp = re.compile(r"\([^\(\)]*\)|;.*|[/\*].*\n")

\printrun\gcoder.py:30: SyntaxWarning: invalid escape sequence '('
m114_exp = re.compile("\([^\(\)]*\)|[/\*].*\n|([XYZ]):?([-+]?[0-9]*\.?[0-9]*)")
--> to: m114_exp = re.compile(r"\([^\(\)]*\)|[/\*].*\n|([XYZ]):?([-+]?[0-9]*\.?[0-9]*)")

\printrun\gcoder.py:31: SyntaxWarning: invalid escape sequence '('
specific_exp = "(?:\([^\(\)]*\))|(?:;.*)|(?:[/\*].*\n)|(%s[-+]?[0-9]*\.?[0-9]*)"
--> to: specific_exp = "(?:\([^\(\)]*\))|(?:;.*)|(?:[/\*].*\n)|(%s[-+]?[0-9]*\.?[0-9]*)"

\printrun\device.py:209: SyntaxWarning: invalid escape sequence '.'
host_regexp = re.compile("^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$|^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$")
--> to: host_regexp = re.compile("^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$|^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$")

\printrun\gcodeplater.py:48: SyntaxWarning: invalid escape sequence '.'
rewrite_exp = re.compile("(%s)" % "|".join(["X([-+]?[0-9]*\.?[0-9]*)",
--> to: rewrite_exp = re.compile("(%s)" % "|".join([r"X([-+]?[0-9]*\.?[0-9]*)",

\printrun\gcodeplater.py:49: SyntaxWarning: invalid escape sequence '.'
"Y([-+]?[0-9]*\.?[0-9]*)"]))
--> to: r"Y([-+]?[0-9]*\.?[0-9]*)"]))

Can't be fixed because external library and do not harm:
\v3\Lib\site-packages\pyglet\gl\wgl.py:36: SyntaxWarning: invalid escape sequence '\c'
"""Wrapper for C:\cygwin\home\Alex\pyglet\tools\wgl.h

@DivingDuck
Copy link
Collaborator Author

For now this PR is work in progress.

@rockstorm101
Copy link
Collaborator

For now this PR is work in progress.

Thank you for such a detailed PR. Let me know when it's ready for review/merge.

@DivingDuck
Copy link
Collaborator Author

Hi @rockstorm101,

Some good news. It looks like we can run Pronterface with Python 3.13 w/o the experimental free-threaded mode as well. 🙂

Plyglet 1.5 became an update some days ago to version 1.5.30 (pyglet/pyglet#1243) which make the 3D visualization working for Python 3.13 again.
I did some tests for all supported python versions from 3.8 up to 3.13 in x32 and x64 with the new version of pyglet to be sure we don't come in trouble.

Test with Windows 10:

Py version x86 x64
3.13.1t no no
3.13.1 ok ok
3.12.8 ok ok
3.11.9 ok ok
3.10.11 ok ok
3.9.13 ok ok
3.8.10 no ok

Latest Windows Artifacts: https://github.com/kliment/Printrun/actions/runs/12306812910

Additional changes for Windows:
release_windows.bat and windows workflow: Remove remark for experimental support for Python 3.13 and add that the experimental Python 3.13t is not supported for now.
release_windows.bat: remove library pypiwin32 and add pywin32 as this version seems to be more supported and compatible with all our supported Python versions for windows. https://pypi.org/project/pywin32/ , https://pypi.org/project/pypiwin32/

MacOs:
macos-12 is now no longer supported. See issue #1461. I removed macos-12 from the macOS workflow and add macos-13, macos-14 and macos-15 for testing.
In addition I remove Python 3.10 as this version isn't compatible with macos-14 and newer. Therefore I add Python 3.11, 3.12 and 3.13 for testing.
Observation from the logs: The workflow step of Build Cython ext contains some warnings.
I can't test those versions as I haven't a Mac available.
Latest macOS Artifacts: https://github.com/kliment/Printrun/actions/runs/12306812915

Build-wheels:
I changed this workflow as well for macOS. Removed macos-12 and add macos-13
Latest build-wheels Artifacts: https://github.com/kliment/Printrun/actions/runs/12306812905
Test needed for Linux as well. Is there something to do in addition for the Linux part?

After testing we need to decide what versions we want to remain for all workflows.

@neofelis2X
Copy link
Contributor

Hello @DivingDuck,
thank you for updating the builds. I tested some of the artifacts on a macOS 15.1.1 and on a macOS 12.7.6

  • Surprisingly, the macos-13 artifacts run on my old macos12 macbook. The later builds do not. That is expected.

The modern macOS is a different story:

  • All artifacts work on macos 15. But macos really does not want to open them.
  • First I get this message and with no option to open them at all:
Screenshot 2024-12-13 at 14 18 44
  • Then I had to dig in the settings to find this option. There was no hint anywhere that I have this option as a user.
Screenshot 2024-12-13 at 14 13 13
  • After clicking on 'Open Anyway' it just works.

That means we will have to improve the instructions on how to install and open Pronterface on mac.

@DivingDuck
Copy link
Collaborator Author

@neofelis2X,
thank you very much taking the time for testing. Indeed, we need to improve the documentation for macOS then.
Guess we should build in addition a matrix of what platform, macOS- and python version works and what not. Something like this if it makes sense:

Platform Python x64 macOS 12.7.6 1 macOS 15.1.1
macos-13 py 3.10 ok ok no
macos-13 py 3.11 ok ok no
macos-13 py 3.12 ok ok no
macos-13 py 3.13 ok ok no
macos-14 py 3.10 no no ?
macos-14 py 3.11 ok no ?
macos-14 py 3.12 ok no ?
macos-14 py 3.13 ok no ?
macos-15 py 3.10 no no no
macos-15 py 3.11 ok no ok
macos-15 py 3.12 ok no ok
macos-15 py 3.13 ok no ok

1 Tested with Macbook

@neofelis2X
Copy link
Contributor

Platform Python x64 macOS 12.7.6 1 macOS 15.1.12
macos-13 py 3.10 ok untested untested
macos-13 py 3.11 ok yes yes
macos-13 py 3.12 ok yes yes
macos-13 py 3.13 ok yes yes
macos-14 py 3.10 no no no
macos-14 py 3.11 ok no yes
macos-14 py 3.12 ok no yes
macos-14 py 3.13 ok no yes
macos-15 py 3.10 no no no
macos-15 py 3.11 ok no yes
macos-15 py 3.12 ok no yes
macos-15 py 3.13 ok no yes

Okay, so that would be the table from my side. Which versions / combinations do you want to release?

I also started to write a little manual on how to handle the security warnings.

Footnotes

  1. Tested on a 2015 Intel Macbook

  2. Tested on a 2022 M2 Macbook

@DivingDuck
Copy link
Collaborator Author

Hi @neofelis2X,

Sorry for my late answer. A flue catch me last week.
Regarding combinations I think we should provide two versions to have a brighter basis for macOS users. MacOS-13 with actual Python 3.13 and macOS-15 with Python 3.13.

Python 3.13 because of Pythons fast development cycle and it seems to be fine for all macOS versions so far.

What do you think about this choice?

Great that you are already start writing a little manual for the installation. +1

I was thinking about Python for the Windows in the same direction - one version with actual version and maybe one with an older version. All other combinations can be build locally from a user as I provide this in the local build script as already shown above.

@rockstorm101,
Any suggestions from your side, especially for Linux platforms?

@rockstorm101
Copy link
Collaborator

Regarding combinations I think we should provide two versions to have a brighter basis for macOS users. MacOS-13 with actual Python 3.13 and macOS-15 with Python 3.13.

For pre-built binaries that are distributed to users I would aim for the lowest version of OS plus the lowest version of Python possible. Because that binary will work (correct me if I'm wrong here) on every OS version above that. Essentially I'm trying to make things simple for users. If one binary works for everyone, just give them one binary and users won't have to decide which binary they have to download. If not, let's try to reduce the number of binaries to the very minimum.

Any suggestions from your side, especially for Linux platforms?

The current recommended way of installing on Linux is via PyPI and there we upload every possible combination so I don't see a debate there. If we fix the pyglet issue, then linux distro packages will create their packages for whatever versions work for them, again, no need for us (upstream) to worry about it I think. So, all good from Linux point of view.

@neofelis2X
Copy link
Contributor

No problem, I was also quite occupied these days. I wish you a speedy recovery then!

I would aim for the lowest version of OS plus the lowest version of Python possible. [...] Essentially I'm trying to make things simple for users.

I generally agree with that, one binary would probably work well for everyone.
However, I did some tests with the versions and compared macos13-python3.11 with macos14-python3.13 : The newer version has a 40% smaller filesize, starts almost 50% faster and also seems to open files about 20% - 30% faster. That are some pretty nice benefits.
On that basis I propose to combine both of your suggestions and release macos13-python3.11 for older macs and macos14-python3.13 for modern macs.

Great that you are already start writing a little manual for the installation. +1

How should I upload it? A separate PR that changes the README? Or as a new file, that we can also include into the released .zip file?

@rockstorm101
Copy link
Collaborator

I propose to [...] release macos13-python3.11 for older macs and macos14-python3.13 for modern macs.

Sounds good to me. Maybe not all macOS 14 users have Python 3.13? Or does it come by default with that Python version?

How should I upload it? A separate PR that changes the README? Or as a new file, that we can also include into the released .zip file?

I'm happy either way. I'd say aim for the README if it is short. Otherwise, a separate file could do. @DivingDuck did some work on GitHub's Wiki pages which could also be an option.

@neofelis2X
Copy link
Contributor

Hi, sorry for the long silence. I got a bit distracted.

Or does it come by default with that Python version?

Python is packaged within the macos application. So it doesnt need to be installed. :)

GitHub's Wiki pages which could also be an option.

Yes, I checked out the wiki and found it to be a good place for this instructions. But now I can not upload my changes because I have no access rights. And wikis don't support regular PRs, it seems?

Can I get rights to upload to the wiki?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants