Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework FileSystem::OpenFile call, and add FILE_FLAGS_NULL_IF_NOT_EXISTS #11297

Merged
merged 33 commits into from
Mar 22, 2024
Merged

Rework FileSystem::OpenFile call, and add FILE_FLAGS_NULL_IF_NOT_EXISTS #11297

Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
6366b86
WIP TryOpenFile again
Mytherin Mar 20, 2024
3d6d096
Compiling again
Mytherin Mar 21, 2024
c7b935a
Merge branch 'main' into tryopenfile2
Mytherin Mar 21, 2024
0ea9d78
Revert WAL changes, skip TPCDS join order test for now
Mytherin Mar 21, 2024
9a18031
Avoid FileExists(path) { RemoveFile(...) } anti-pattern and replace w…
Mytherin Mar 21, 2024
56162d5
Remove TryOpenFile, instead make this a parameter to OpenFile
Mytherin Mar 21, 2024
eee76b8
Rework OpenFile - merge lock and compression into FileOpenFlags
Mytherin Mar 21, 2024
32997ab
Fix httpfs compilation
Mytherin Mar 21, 2024
be0d0b0
Improve documentation of S3 tests
Mytherin Mar 21, 2024
e92709c
Warn about concurrently running clickhouse, and ignore errors in dock…
Mytherin Mar 21, 2024
0c4c848
DynamicCastCheck
Mytherin Mar 21, 2024
4b4081d
Revert changes to RemoveFile/RemoveDirectory
Mytherin Mar 21, 2024
2aac450
Fix #9342: WAL - instead of using FileExists -> OpenFile, use OpenFil…
Mytherin Mar 21, 2024
e9d8a86
Format fix
Mytherin Mar 21, 2024
0d29350
Fix for pyfilesystem
Mytherin Mar 21, 2024
cd5a647
FILE_FLAGS_NULL_IF_NOT_EXISTS cannot be combined with CREATE
Mytherin Mar 21, 2024
816c7eb
Add azure/spatial patches
Mytherin Mar 21, 2024
1dba9b5
APPLY_PATCHES for mingw
Mytherin Mar 21, 2024
bd6a157
Actually fix Azure and Spatial
Mytherin Mar 21, 2024
2982b5f
Format fix
Mytherin Mar 21, 2024
ba9fd81
Rename to more logical name
Mytherin Mar 21, 2024
7b3917d
Add more documentation here
Mytherin Mar 21, 2024
f9b89b6
Fix for Windows
Mytherin Mar 21, 2024
a55feb3
Run generate_enum_util again
Mytherin Mar 21, 2024
55c35fd
Implement FILE_FLAGS_NULL_IF_NOT_EXISTS in pyfilesystem
Mytherin Mar 21, 2024
2b8e25b
Format
Mytherin Mar 21, 2024
122634b
path_p
Mytherin Mar 21, 2024
f09ad43
Avoid checking for the existence of :memory:, and avoid unnecessarily…
Mytherin Mar 21, 2024
24f343d
Keep using fs.FileExists for extensions
Mytherin Mar 21, 2024
5ae0f61
Fix iceberg
Mytherin Mar 21, 2024
5617ee0
Merge branch 'main' into tryopenfile2
Mytherin Mar 21, 2024
b0f55ee
Re-enable this test
Mytherin Mar 21, 2024
8837144
Windows spatial fix
Mytherin Mar 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Windows spatial fix
  • Loading branch information
@Mytherin
Mytherin committed Mar 22, 2024
commit 8837144de160639c2cf0851ad12703c25e263e8d
12 changes: 10 additions & 2 deletions .github/patches/extensions/spatial/open_file.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ index dfe396a..87bf598 100644

auto max_threads = context.db->NumberOfThreads();
diff --git a/spatial/src/spatial/gdal/file_handler.cpp b/spatial/src/spatial/gdal/file_handler.cpp
index db449df..4985c9f 100644
index db449df..ebcefe5 100644
--- a/spatial/src/spatial/gdal/file_handler.cpp
+++ b/spatial/src/spatial/gdal/file_handler.cpp
@@ -122,7 +122,7 @@ public:
Expand All @@ -24,7 +24,15 @@ index db449df..4985c9f 100644
auto len = strlen(access);
if (access[0] == 'r') {
flags = FileFlags::FILE_FLAGS_READ;
@@ -167,7 +167,7 @@ public:
@@ -160,14 +160,14 @@ public:
// Check if the file is a directory

#ifdef _WIN32
- if (fs.DirectoryExists(path) && (flags & FileFlags::FILE_FLAGS_READ)) {
+ if (fs.DirectoryExists(path) && flags.OpenForReading()) {
// We can't open a directory for reading on windows without special flags
// so just open nul instead, gdal will reject it when it tries to read
auto file = fs.OpenFile("nul", flags);
return new DuckDBFileHandle(std::move(file));
}
#endif
Expand Down
Loading