Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Go bindings + macOS: warning: 'sqlite3_auto_extension' is deprecated #169

Open
mholt opened this issue Jan 13, 2025 · 2 comments
Open

Go bindings + macOS: warning: 'sqlite3_auto_extension' is deprecated #169

mholt opened this issue Jan 13, 2025 · 2 comments

Comments

@mholt
Copy link

mholt commented Jan 13, 2025

(Copying this from Discord as suggested)

I believe the Go bindings for this use sqlite3_auto_extension, which is cool, but on Mac, I get this deprecation warning:

# github.com/asg017/sqlite-vec-go-bindings/cgo
cgo-gcc-prolog:55:11: warning: 'sqlite3_auto_extension' is deprecated: first deprecated in macOS 10.10 - Process-global auto extensions are not supported on Apple platforms [-Wdeprecated-declarations]
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sqlite3.h:7085:16: note: 'sqlite3_auto_extension' has been explicitly marked deprecated here
cgo-gcc-prolog:74:11: warning: 'sqlite3_cancel_auto_extension' is deprecated: first deprecated in macOS 10.14 - Process-global auto extensions are not supported on Apple platforms [-Wdeprecated-declarations]
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sqlite3.h:7098:16: note: 'sqlite3_cancel_auto_extension' has been explicitly marked deprecated here

@asg017 replied:

I've gotten this deprecation warning too, but the build still succeeds. So it's 1 of 2 things happening:

  1. go-sqlite3 uses MacOS SQLite, and since sqlite3_auto_extension() is deprecated but not removed, the build still works, but complains. But a future version of MacOS might break this?
  2. The go-sqlite3 build process references /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sqlite3.h , which causes the deprecation warning, but still uses it's bundled version of SQLite in the actual build process, which doesn't include the deprecated sqlite3_auto_extension() function and build successfully

I'm not sure which one is right, tbh. Because the MacOS SQLite also completely blocks sqlite3_load_extension(), yet go-sqlite3 can still dynamically load SQLite extensions, so # 2 seems more likely, but the Go/SQLite compilation process is complicated so I cant say for sure.

That being said, I don't like relying on sqlite3_auto_extension(). Technically, you could call sqlite3_vec_init(db, NULL, NULL) yourself in C to load the sqlite-vec extension on a database connection, and technically this could be done in Go with the ConnectHook mechanism. But when I tried last year, I got segfaults and couldn't get it to work reliably.

That being said, if the .Auto() mechanism doesnt work, or if you can't have the delcartion warnings, feel free to file and issue and I'll take another work! Just copying+pasting this message should work 💯

Which makes sense. I don't know a lot about it though.

Then, I thought:

I'm wondering, since it seems like a good idea to heed the warning at some point... is it possible to use something like this, which I did for a ulid extension in a project one time?

sql.Register("sqlite3-ulid", &sqlite3.SQLiteDriver{
    ConnectHook: func(conn *sqlite3.SQLiteConn) error {
        return conn.LoadExtension("/home/matt/Downloads/ulid0.so", "sqlite3_ulid_init")
    },
})
@asg017
Copy link
Owner

asg017 commented Jan 13, 2025

Re the conn.LoadExtension() approach - that totally works, but the reason why I provide the Go bindings is so developers can statically link sqlite-vec directly into their compiled Go binary. The dynamic loading approach requires you to manage a separate loadable binary in addition to your main executable Go binary, which can be annoying. Same with the C/Rust bindings, you could also dynamically load sqlite-vec as a separate loadable extension, but static linking is always nicer.

That being said, we should still be able to provide static linking without using sqlite3_auto_extension(), which works but is a bit awkward, and raises warnings in MacOS builds.

Ideally we should be able to call the sqlite3_vec_init() C function directly in ConnectHook(), but I wasn't able to get that to work in the past. I'll give it another shot soon!

@mholt
Copy link
Author

mholt commented Jan 14, 2025

Ah yeah, that makes sense I guess; and the autoext is really nice in that sense.

I'm just trying to clean up my compilation output and future-proof the application a bit more.

I maaaay still opt to just load the extension dynamically, although I guess that means I need to ship a separate file now. 😅

But yeah, if there's a way to statically link it without having compilation warnings, that'd be ideal 👌

Thanks so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants