Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.12 - AttributeError: module 'pathlib' has no attribute '_Flavour' #430

Open
briantist opened this issue Aug 13, 2023 · 44 comments
Open
Labels
enhancement Feature request to extend functionality hacktoberfest Help Wanted We will be glad if somebody proposes a solution via PR

Comments

@briantist
Copy link
Contributor

briantist commented Aug 13, 2023

Python 3.12 is now released, but it does not work due to pathlib changes:

/opt/hostedtoolcache/Python/3.12.0-rc.1/x64/lib/python3.12/site-packages/artifactory.py:426: in
class _ArtifactoryFlavour(pathlib._Flavour):
E AttributeError: module 'pathlib' has no attribute '_Flavour'

I figured I'd add 3.12 into the test matrix on my project which relies on this library, and that's how I found this issue:

Not any kind of pressing issue yet.

@allburov allburov added the enhancement Feature request to extend functionality label Aug 14, 2023
@allburov
Copy link
Member

Thank you for the feedback!
If anyone has idea how to fix this - feel free to create a PR

@brot
Copy link

brot commented Sep 4, 2023

Python 3.12 will add support for subclassing pathlib.PurePath and Path

See also the cpython github issue: python/cpython#31691

@barneygale
Copy link

I'm hoping to add a pathlib.PathBase class for Python 3.13. Once it's in the codebase, I'll publish a backport package on PyPI for Python <3.13. dohq-artifactory could subclass PathBase to implement ArtifactoryPath.

CPython issue: python/cpython#110109

@allburov allburov added the Help Wanted We will be glad if somebody proposes a solution via PR label Oct 10, 2023
@briantist briantist changed the title Python 3.12 (pre-release) - AttributeError: module 'pathlib' has no attribute '_Flavour' Python 3.12 - AttributeError: module 'pathlib' has no attribute '_Flavour' Nov 20, 2023
@ktdreyer
Copy link
Contributor

ktdreyer commented Dec 8, 2023

@barneygale Thank you for your work in core for this. It will be great to simplify the dohq-artifactory codebase.

Do you suggest any other solution for py312 in the meantime? Should we try subclassing something from py312's pathlib prior to pathlib.PathBase being available?

I'm running Python 3.12 on my laptop (Fedora 39), and I've hacked artifactory.py so that it doesn't crash. I can list directories and download and upload basic files. My changes are in my py312 branch, but I'm not sure they're acceptable for a PR here.

@barneygale
Copy link

I'd recommend that you wait if you can - I'm expecting to publish a PyPI package with the base classes some time this month.

@barneygale
Copy link

Git project: https://github.com/barneygale/pathlib_abc

Currently it only supports Python 3.11+, but I should have it working for 3.8+ within a couple of weeks, at which point I'll publish a PyPI package.

@barneygale
Copy link

First PyPI release: https://pypi.org/project/pathlib-abc/0.1.0/

@FHTMitchell
Copy link

I'm seeing the same issue in py3.11 now

@dkmiller
Copy link

dkmiller commented Feb 5, 2024

Hey everyone - any progress here?

@zhan9san
Copy link
Contributor

zhan9san commented Feb 6, 2024

Hey guys,

Thanks for reporting this issue. I'd like to work on it.

@barneygale
Copy link

barneygale commented Feb 24, 2024

IMO the best way to fix this is to add a dependency on pathlib-abc, and make ArtifactoryPath inherit from PathBase.

@zhan9san is that what you're pursuing? If not I might have a crack at it.

@lukezhangtyler
Copy link

I hope this will be remediated soon.

@okainov
Copy link

okainov commented Mar 19, 2024

@zhan9san @beliaev-maksim hi folks, any update on this issue? Given Python 3.12 is fully released and supported, it would be very appreciated that Artifactory library supports it, it's the only item blocking our migration to 3.12 now...

@allburov
Copy link
Member

We are looking for a volunteer!

@andrewsiemer
Copy link

+1 this is blocking migrating to python 3.12

@hacklint
Copy link

hacklint commented May 3, 2024

Any hope of this being resolved before release of python 3.13?

@barneygale
Copy link

FWIW I've started a new pathlib-artifactory package building on the semi-official pathlib-abc package. It doesn't rely on hacking into the pathlib internals (unlike dohq-artifactory) so it should be on surer footing. But it's missing tons of features and it's certainly not a drop-in replacement for dohq-artifactory yet. Issues and PRs very welcome.

@seltsa
Copy link

seltsa commented Sep 10, 2024

Any updates for Fedora39 ? Hit the same issue

@LeiYangGH
Copy link

FWIW I've started a new pathlib-artifactory package building on the semi-official pathlib-abc package. It doesn't rely on hacking into the pathlib internals (unlike dohq-artifactory) so it should be on surer footing. But it's missing tons of features and it's certainly not a drop-in replacement for dohq-artifactory yet. Issues and PRs very welcome.

thanks and could you add a little document on the readme, for example, why create the project, are the usage compatible with devopshq artifactory?

@FrogWithASock
Copy link

Any updates? Is this project dead? There was no commit to main for 7 months now

@allburov
Copy link
Member

Is this project dead?

Not dead, but yeah, no active maintainer here :(
We're happy to review any PR for the issue and any help! 🙏

I don't work with artifactory anymore, so can not even test it :(

@oscarm3l1n
Copy link

@barneygale Thank you for your work in core for this. It will be great to simplify the dohq-artifactory codebase.

Do you suggest any other solution for py312 in the meantime? Should we try subclassing something from py312's pathlib prior to pathlib.PathBase being available?

I'm running Python 3.12 on my laptop (Fedora 39), and I've hacked artifactory.py so that it doesn't crash. I can list directories and download and upload basic files. My changes are in my py312 branch, but I'm not sure they're acceptable for a PR here.

Tested this locally, works like a charm!

@oscarm3l1n
Copy link

@barneygale Thank you for your work in core for this. It will be great to simplify the dohq-artifactory codebase.

Do you suggest any other solution for py312 in the meantime? Should we try subclassing something from py312's pathlib prior to pathlib.PathBase being available?

I'm running Python 3.12 on my laptop (Fedora 39), and I've hacked artifactory.py so that it doesn't crash. I can list directories and download and upload basic files. My changes are in my py312 branch, but I'm not sure they're acceptable for a PR here.

@ktdreyer
Since it's been almost a year. Have you worked around this issue somehow? (sounded on your implementation that it was more of a hack)

@RoccoMatano
Copy link

I use a little application that depends on artifactory and I want to run it with current python versions (i.e. 3.13). I don't know anything about the internals of artifactory, but I managed to patch it in such a way, that it fullfills my needs. I have no idea whether this works for all use cases.
For what it's worth, here's the diff:

--- C:\prj\progs\py313\Lib\site-packages\artifactory.0.10.1.py	2024-10-08 08:10:50.000000000 +0200
+++ C:\prj\progs\py313\Lib\site-packages\artifactory.py	2024-10-08 08:16:13.000000000 +0200
@@ -419,13 +419,13 @@
         quoted_path = requests.utils.quote(url.partition(parsed_url.host)[2])
         quoted_url = f"{parsed_url.scheme}://{parsed_url.host}{quoted_path}"
 
     return quoted_url
 
 
-class _ArtifactoryFlavour(pathlib._Flavour):
+class _ArtifactoryFlavour:
     """
     Implements Artifactory-specific pure path manipulations.
     I.e. what is 'drive', 'root' and 'path' and how to split full path into
     components.
     See 'pathlib' documentation for explanation how those are used.
 
@@ -437,13 +437,12 @@
     path: relative artifact path within the repository
     """
 
     sep = "/"
     altsep = "/"
     has_drv = True
-    pathmod = pathlib.posixpath
     is_supported = True
 
     def _get_base_url(self, url):
         return get_global_base_url(url)
 
     def compile_pattern(self, pattern):
@@ -1498,25 +1497,33 @@
     """
 
     if sys.version_info.major == 3 and sys.version_info.minor >= 10:
         # see changes in pathlib.Path, slots are no more applied
         # https://github.com/python/cpython/blob/ce121fd8755d4db9511ce4aab39d0577165e118e/Lib/pathlib.py#L952
         _accessor = _artifactory_accessor
+        parser = _ArtifactoryFlavour()
     else:
         # in 3.9 and below Pathlib limits what members can be present in 'Path' class
         __slots__ = ("auth", "verify", "cert", "session", "timeout")
+
+    def __init__(self, *args, **kwargs):
+        # supplying keyword arguments to pathlib.PurePath is deprecated
+        return super().__init__(*args)
 
     def __new__(cls, *args, **kwargs):
         """
         pathlib.Path overrides __new__ in order to create objects
         of different classes based on platform. This magic prevents
         us from adding an 'auth' argument to the constructor.
         So we have to first construct ArtifactoryPath by Pathlib and
         only then add auth information.
         """
         obj = pathlib.Path.__new__(cls, *args, **kwargs)
+        if sys.version_info.major == 3 and sys.version_info.minor >= 12:
+            # supplying keyword arguments to pathlib.PurePath is deprecated
+            obj.__init__(*args)
 
         cfg_entry = get_global_config_entry(obj.drive)
 
         # Auth section
         apikey = kwargs.get("apikey")
         token = kwargs.get("token")

@hacklint
Copy link

I use a little application that depends on artifactory and I want to run it with current python versions (i.e. 3.13). I don't know anything about the internals of artifactory, but I managed to patch it in such a way, that it fullfills my needs. I have no idea whether this works for all use cases. For what it's worth, here's the diff:

--- C:\prj\progs\py313\Lib\site-packages\artifactory.0.10.1.py	2024-10-08 08:10:50.000000000 +0200
+++ C:\prj\progs\py313\Lib\site-packages\artifactory.py	2024-10-08 08:16:13.000000000 +0200
@@ -419,13 +419,13 @@
         quoted_path = requests.utils.quote(url.partition(parsed_url.host)[2])
         quoted_url = f"{parsed_url.scheme}://{parsed_url.host}{quoted_path}"
 
     return quoted_url
 
 
-class _ArtifactoryFlavour(pathlib._Flavour):
+class _ArtifactoryFlavour:
     """
     Implements Artifactory-specific pure path manipulations.
     I.e. what is 'drive', 'root' and 'path' and how to split full path into
     components.
     See 'pathlib' documentation for explanation how those are used.
 
@@ -437,13 +437,12 @@
     path: relative artifact path within the repository
     """
 
     sep = "/"
     altsep = "/"
     has_drv = True
-    pathmod = pathlib.posixpath
     is_supported = True
 
     def _get_base_url(self, url):
         return get_global_base_url(url)
 
     def compile_pattern(self, pattern):
@@ -1498,25 +1497,33 @@
     """
 
     if sys.version_info.major == 3 and sys.version_info.minor >= 10:
         # see changes in pathlib.Path, slots are no more applied
         # https://github.com/python/cpython/blob/ce121fd8755d4db9511ce4aab39d0577165e118e/Lib/pathlib.py#L952
         _accessor = _artifactory_accessor
+        parser = _ArtifactoryFlavour()
     else:
         # in 3.9 and below Pathlib limits what members can be present in 'Path' class
         __slots__ = ("auth", "verify", "cert", "session", "timeout")
+
+    def __init__(self, *args, **kwargs):
+        # supplying keyword arguments to pathlib.PurePath is deprecated
+        return super().__init__(*args)
 
     def __new__(cls, *args, **kwargs):
         """
         pathlib.Path overrides __new__ in order to create objects
         of different classes based on platform. This magic prevents
         us from adding an 'auth' argument to the constructor.
         So we have to first construct ArtifactoryPath by Pathlib and
         only then add auth information.
         """
         obj = pathlib.Path.__new__(cls, *args, **kwargs)
+        if sys.version_info.major == 3 and sys.version_info.minor >= 12:
+            # supplying keyword arguments to pathlib.PurePath is deprecated
+            obj.__init__(*args)
 
         cfg_entry = get_global_config_entry(obj.drive)
 
         # Auth section
         apikey = kwargs.get("apikey")
         token = kwargs.get("token")

Tried this locally, worked fine for what I tested (read-only from Artifactory, no writes tested)

@LeiYangGH
Copy link

I use a little application that depends on artifactory and I want to run it with current python versions (i.e. 3.13). I don't know anything about the internals of artifactory, but I managed to patch it in such a way, that it fullfills my needs. I have no idea whether this works for all use cases. For what it's worth, here's the diff:

--- C:\prj\progs\py313\Lib\site-packages\artifactory.0.10.1.py	2024-10-08 08:10:50.000000000 +0200
+++ C:\prj\progs\py313\Lib\site-packages\artifactory.py	2024-10-08 08:16:13.000000000 +0200
@@ -419,13 +419,13 @@
         quoted_path = requests.utils.quote(url.partition(parsed_url.host)[2])
         quoted_url = f"{parsed_url.scheme}://{parsed_url.host}{quoted_path}"
 
     return quoted_url
 
 
-class _ArtifactoryFlavour(pathlib._Flavour):
+class _ArtifactoryFlavour:
     """
     Implements Artifactory-specific pure path manipulations.
     I.e. what is 'drive', 'root' and 'path' and how to split full path into
     components.
     See 'pathlib' documentation for explanation how those are used.
 
@@ -437,13 +437,12 @@
     path: relative artifact path within the repository
     """
 
     sep = "/"
     altsep = "/"
     has_drv = True
-    pathmod = pathlib.posixpath
     is_supported = True
 
     def _get_base_url(self, url):
         return get_global_base_url(url)
 
     def compile_pattern(self, pattern):
@@ -1498,25 +1497,33 @@
     """
 
     if sys.version_info.major == 3 and sys.version_info.minor >= 10:
         # see changes in pathlib.Path, slots are no more applied
         # https://github.com/python/cpython/blob/ce121fd8755d4db9511ce4aab39d0577165e118e/Lib/pathlib.py#L952
         _accessor = _artifactory_accessor
+        parser = _ArtifactoryFlavour()
     else:
         # in 3.9 and below Pathlib limits what members can be present in 'Path' class
         __slots__ = ("auth", "verify", "cert", "session", "timeout")
+
+    def __init__(self, *args, **kwargs):
+        # supplying keyword arguments to pathlib.PurePath is deprecated
+        return super().__init__(*args)
 
     def __new__(cls, *args, **kwargs):
         """
         pathlib.Path overrides __new__ in order to create objects
         of different classes based on platform. This magic prevents
         us from adding an 'auth' argument to the constructor.
         So we have to first construct ArtifactoryPath by Pathlib and
         only then add auth information.
         """
         obj = pathlib.Path.__new__(cls, *args, **kwargs)
+        if sys.version_info.major == 3 and sys.version_info.minor >= 12:
+            # supplying keyword arguments to pathlib.PurePath is deprecated
+            obj.__init__(*args)
 
         cfg_entry = get_global_config_entry(obj.drive)
 
         # Auth section
         apikey = kwargs.get("apikey")
         token = kwargs.get("token")

Tried this locally, worked fine for what I tested (read-only from Artifactory, no writes tested)

how to apply this patch after installing the official module?
it would be great if we can copy the patch to our project source folder which then 'monkey patch' the installed official module. is it possible?

@allburov
Copy link
Member

patching is not the way to go, we should add it as built-in fix (if it works for all cases)

Let's not discuss patching here, I'll mark the comment as Off Topic in the future

okainov added a commit to okainov/artifactory that referenced this issue Nov 4, 2024
@okainov
Copy link

okainov commented Nov 4, 2024

For what it's worth, here's the diff:

@RoccoMatano FYI I've created PR based on your patch #458

@Gris87
Copy link

Gris87 commented Nov 11, 2024

Hi, any progress with this issue?

@allburov
Copy link
Member

allburov commented Jan 13, 2025

Hi all participants, we need your help! 🙏

Could you test the latest changes in python 3.12 and your script/logic?
It has some backward incompatible changes, but likely most of the usage will not be affected
#463

Here's how you can install it:

pip install git+https://github.com/auburus/dohq-artifactory@fix/python-312-compat-from-refactor-branch

Waiting for your feedback (both positive and some issues if you find any!)

@axel-kah
Copy link

axel-kah commented Jan 13, 2025

Could you test the latest changes in python 3.12 and your script/logic? It has some backward incompatible changes, but likely most of the usage will not be affected #463

Tested my script, which uses ArtifactoryPath, RepositoryLocal, RepositoryRemote, with your PR:

  • 3.11 ✅
  • 3.12 ✅

Thanks to @auburus for putting in time and energy to help make this lib not a legacy-trap 👍

EDIT: Artifactory version is Enterprise Plus 7.63.14 rev 76314900

@doronz88
Copy link
Contributor

Does this PR also fix python3.13 support?

@axel-kah
Copy link

axel-kah commented Jan 13, 2025

It's a bit tricky to test for my usecases because this package is a transient dependency in many cases but smoke testing with python 3.12, in those cases where I can tweak the dependencies, indicates it works just like expected.

Only thing of note was a deprecation warning for 3.14:

  C:\dev\foo\.hatch\foo\Lib\site-packages\artifactory.py:1618: DeprecationWarning: support for supplying keyword arguments to pathlib.PurePath is deprecated and scheduled for removal in Python 3.14
    super().__init__(*args, **kwargs)

@ANVLISTENER
Copy link

Hi Everyone,

When can we expect this to be tested clearly and be merged. We're literally waiting of this to happen as there are blockers in Ubuntu 24.04 depending on python3.12 completely hardcoded in the OS.

Once, we merge we would be in relief.

Please fix and merge it if it fixes the issue ASAP.

humble request please.

thank you.

@johannesloibl
Copy link

johannesloibl commented Jan 15, 2025

Hi Everyone,

When can we expect this to be tested clearly and be merged. We're literally waiting of this to happen as there are blockers in Ubuntu 24.04 depending on python3.12 completely hardcoded in the OS.

Once, we merge we would be in relief.

Please fix and merge it if it fixes the issue ASAP.

humble request please.

thank you.

@ANVLISTENER You could help testing it and give feedback in order to help merging this fix "ASAP" ;)

@ANVLISTENER
Copy link

I hope we can have a clear support in the future for future releases of python.

I hope easy way is to restrict a version to support with transient fix and later versions with pure support later( alternate native code as per python 3.13 or later).

we will need see it in the slightly middle future kind of by mid or later in 2025. it is a blocker in the real time implementations In our use case with artifactory as dependency.

@ANVLISTENER
Copy link

ANVLISTENER commented Jan 15, 2025

@johannesloibl I have limited knowledge on this and as per little feedback it works clean for our use case.

This was the response:

I have confirmed that this fix does work for Ubuntu 24.

It should ideally be universal as well as it a platform support rather than any code testing use case.

@axel-kah
Copy link

I hope we can have a clear support in the future for future releases of python.

[...]

we will need see it in the slightly middle future kind of by mid or later in 2025. it is a blocker in the real time implementations In our use case with artifactory as dependency.

You do realize this is an open source project with no funding and limited commitment from the maintainer ?

Don't get me wrong, this is the best artifactory python lib I know of, but if it's that important to you or your business, then treat it as such and invest energy/time/money to keep it healthy or manage your expectations.

@ANVLISTENER
Copy link

ANVLISTENER commented Jan 15, 2025

@axel-kah I understand the perspective, but the wish is this to be have a fix and be make it better.

Shared my thoughts to it's best it should be and as I am aware of this is the best artifactory python lib I know of as said.

The treasure work of #463 for time being works and is too good

@ANVLISTENER
Copy link

ANVLISTENER commented Jan 15, 2025

@axel-kah It works fine and I did test for this issue which is no more.

@allburov
Copy link
Member

Shouts to @auburus for the hard work and the fix, so far it's working for all cases, as I can understand.

Let's set a deadline - 21 January 2025. If no critical bugs found before that date - we'll merge #463 and make a release 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Feature request to extend functionality hacktoberfest Help Wanted We will be glad if somebody proposes a solution via PR
Development

No branches or pull requests