You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
i would be interested in having the equivalent of writeto but for upload. I would like to implement a tdqm progress bar for long upload, and for that i need somehow not to be blocked during deploy_file, or provide a callback mecanism.
is it something possible to do right now?
Here is my code for the download progress using tdqm:
CHUNK_SIZE=1024*1024# 1MBdefdownload_file_tqdm(
source_file: ArtifactoryPath,
target_file: Path,
chuck_size: int=CHUNK_SIZE,
progress_bar_desc: str="Downloading",
):
""" Download a file from Artifactory with a tqdm progress bar. Args: source_file: The source file path in Artifactory. target_file: The target file path on the local filesystem. Parent folders are created. chuck_size: The size of each chunk to read at a time. Defaults to 1 Mb. progress_bar_desc: The description of the progress bar. Defaults to "Downloading". Example: .. code-block:: python source = ArtifactoryPath("http://artifactory.example.com/path/to/file") target = Path("/local/path/to/file") download_file_tqdm(source, target) """class_TdqmArtifactoryPathWriteTo(tqdm.tqdm):
defprogress_func(self, bytes_now, total_size):
iftotal_sizeisnotNone:
self.total=total_sizereturnself.update(bytes_now-self.n) # also sets self.n = b * bsizetarget_file.parent.mkdir(parents=True, exist_ok=True)
withopen(str(target_file), "wb") astarget_fd:
with_TdqmArtifactoryPathWriteTo(
desc=progress_bar_desc,
unit="B",
unit_scale=True,
unit_divisor=1024,
miniters=1,
) ast:
source_file.writeto(
target_fd,
chunk_size=chuck_size,
progress_func=t.progress_func,
)
thanks !
The text was updated successfully, but these errors were encountered:
defupload_file_tqdm(
local_file_path: Path,
remote_path: ArtifactoryPath,
chuck_size: int=CHUNK_SIZE,
progress_bar_desc: str="Uploading",
):
""" Upload a file to Artifactory with a progress bar. Args: local_file_path: The local file path to upload. remote_path: The remote Artifactory path where the file will be uploaded. Existing file will be replaced. chuck_size: The size of each chunk to read at a time. Defaults to 1 Mb. progress_bar_desc: The description of the progress bar. Defaults to "Uploading". Example: .. code-block:: python local_file = Path("/local/path/to/file") remote = ArtifactoryPath("http://artifactory.example.com/path/to/file") upload_file_tqdm(local_file, remote) """ifnotlocal_file_path.exists():
raiseFileNotFoundError(f"File {local_file_path} does not exist")
filesize=local_file_path.stat().st_sizeiffilesize==0:
# File is empty, just create a remote, empty fileremote_path.touch()
returnwithlocal_file_path.open("rb") asfobj:
withtqdm.tqdm(
desc=progress_bar_desc,
total=filesize,
unit_scale=True,
unit="B",
unit_divisor=1024,
miniters=1,
) ast:
defiter_in_file():
forchunkinread_in_chunks(
fobj,
chunk_size=chuck_size,
):
yieldchunkt.update(len(chunk))
session=remote_path.sessionsession.put(
str(remote_path),
data=iter_in_file(),
stream=True,
)
Hello,
i would be interested in having the equivalent of
writeto
but for upload. I would like to implement a tdqm progress bar for long upload, and for that i need somehow not to be blocked duringdeploy_file
, or provide a callback mecanism.is it something possible to do right now?
Here is my code for the download progress using tdqm:
thanks !
The text was updated successfully, but these errors were encountered: