What's Changed

Patch release with bug fixes. No breaking changes.

Bug fixes

• HTTP timeout wasn't always being correctly applied in the downstream c-questdb-client dependency.
• request_timeout > 0 is be enforced. This was always required, but would not error.
• Fixed the source distribution "sdist" package: This allows the package to be installed from source via "pip install" on previously unsupported platforms (YMMV).

Full Changelog: v2.0.2...v2.0.3

What's Changed

Patch release with performance bug fix. No breaking changes.

Bug fixes

• Fixed the defaulting logic for auto_flush_rows parameter for HTTPS.
  It is now correctly set to 75000 rows by default. The old incorrect default
  of 600 rows was causing the sender to flush too often, impacting performance.
  Note that TCP, TCPS and HTTP were not affected.

Features

• The sender now exposes the auto_flush settings as read-only properties.
  You can inspect the values in use with .auto_flush, .auto_flush_rows,
  .auto_flush_interval and .auto_flush_bytes.

Full Changelog: v2.0.1...v2.0.2

What's Changed

This is a patch release with bug fixes, no API changes and some documentation tweaks.

Bug fixes

• Fixed a bug where an internal "last flushed" timestamp used by auto_flush_interval wasn't updated correctly causing the auto-flush logic to trigger after each row.
• Removed two unnecessary debugging print() statements that were accidentally left in the code in Sender.from_conf() and
  Sender.from_env().

Improvements

• Introduced the ability to optionally install pandas and pyarrow via python3 -m pip install -U questdb[dataframe] and updated the documentation to reflect this.

What's Changed

This is a major release with new features and breaking changes.

Features

• Support for ILP over HTTP. The sender can now send data to QuestDB via HTTP instead of TCP. This provides error feedback from the server and new features.

  conf = 'http::addr=localhost:9000;'
  with Sender.from_conf(conf) as sender:
      sender.row(...)
      sender.dataframe(...)
  
      # Will raise `IngressError` if there is an error from the server.
      sender.flush()

• New configuration string construction. The sender can now be also constructed from a configuration string in addition to the constructor arguments. This allows for more flexible configuration and is the recommended way to construct a sender. The same string can also be loaded from the QDB_CLIENT_CONF environment variable. The constructor arguments have been updated and some options have changed.

• Explicit transaction support over HTTP. A set of rows for a single table can now be committed via the sender transactionally. You can do this using a with sender.transaction('table_name') as txn: block.

  conf = 'http::addr=localhost:9000;'
  with Sender.from_conf(conf) as sender:
      with sender.transaction('test_table') as txn:
          # Same arguments as the sender methods, minus the table name.
          txn.row(...)
          txn.dataframe(...)

• A number of documentation improvements.

Breaking Changes

• New protocol parameter in the Sender constructor. In the previous version, the protocol was always TCP. In this new version, you must specify the protocol explicitly.

• New auto-flush defaults. In previous versions, auto-flushing was enabled by default and triggered by a maximum buffer size. In this new version, auto-flushing is enabled by row count (600 rows by default) and interval (1 second by default), while auto-flushing by buffer size is disabled by default.

  The old behavior can still be achieved by tweaking the auto-flush settings.

  Setting	Old default	New default
  auto_flush_rows	off	600
  auto_flush_interval	off	1000
  auto_flush_bytes	64512	off

• The at=.. argument of the row and dataframe methods is now mandatory. Omitting it would previously use a server-generated timestamp for the row. Now, if you want a server-generated timestamp, you can pass the ServerTimestamp singleton to this parameter. The ServerTimestamp behavior is considered legacy.

• The auth=(u, t, x, y) argument of the Sender constructor has now been broken up into multiple arguments: username, token, token_x, token_y.

• The tls argument of the Sender constructor has been removed and replaced with the protocol argument. Use Protocol.Tcps (or Protocol.Https) to enable TLS. The tls values have been moved to new tls_ca and tls_roots configuration settings.

• The net_interface argument of the Sender constructor has been renamed to bind_interface and is now only available for TCP connections.

The following example shows how to migrate to the new API.

Old questdb 1.x code

from questdb.ingress import Sender

auth = (
    'testUser1', 
    '5UjEMuA0Pj5pjK8a-fa24dyIf-Es5mYny3oE_Wmus48',
    'token_x=fLKYEaoEb9lrn3nkwLDA-M_xnuFOdSt9y0Z7_vWSHLU',
    'token_y=Dt5tbS1dEDMSYfym3fgMv0B99szno-dFc1rYF9t0aac')
with Sender('localhost', 9009, auth=auth, tls=True) as sender:
    sender.row(
        'test_table',
        symbols={'sym': 'AAPL'},
        columns={'price': 100.0})  # `at=None` was defaulted for server time

Equivalent questdb 2.x code

from questdb.ingress import Sender, Protocol, ServerTimestamp

sender = Sender(
    Protocol.Tcps,
    'localhost',
    9009,
    username='testUser1',
    token='5UjEMuA0Pj5pjK8a-fa24dyIf-Es5mYny3oE_Wmus48',
    token_x='token_x=fLKYEaoEb9lrn3nkwLDA-M_xnuFOdSt9y0Z7_vWSHLU',
    token_y='token_y=Dt5tbS1dEDMSYfym3fgMv0B99szno-dFc1rYF9t0aac',
    auto_flush_rows='off',
    auto_flush_interval='off',
    auto_flush_bytes=64512)
with sender:
    sender.row(
        'test_table',
        symbols={'sym': 'AAPL'},
        columns={'price': 100.0},
        at=ServerTimestamp)

Equivalent questdb 2.x code with configuration string

from questdb.ingress import Sender, ServerTimestamp

conf = (
    'tcp::addr=localhost:9009;' +
    'username=testUser1;' +
    'token=5UjEMuA0Pj5pjK8a-fa24dyIf-Es5mYny3oE_Wmus48;' +
    'token_x=token_x=fLKYEaoEb9lrn3nkwLDA-M_xnuFOdSt9y0Z7_vWSHLU;' +
    'token_y=token_y=Dt5tbS1dEDMSYfym3fgMv0B99szno-dFc1rYF9t0aac;' +
    'auto_flush_rows=off;' +
    'auto_flush_interval=off;' +
    'auto_flush_bytes=64512;')
with Sender.from_conf(conf) as sender:
    sender.row(
        'test_table',
        symbols={'sym': 'AAPL'},
        columns={'price': 100.0},
        at=ServerTimestamp)

What's Changed

This is a minor release bringing in minor new features and a few bug fixes, without any breaking changes.
Most changes are inherited by internally upgrading to version 3.1.0 of the c-questdb-client.

Features

• Sender(..., tls=True) now also uses OS-provided certificate store. The tls argument can now also be set to tls='os_roots' (to only use the OS-provided certs) or tls='webpki_roots' (to only use the certs provided by the webpki-roots, i.e. the old behaviour prior to this release). The new default behaviour for tls=True is equivalent to setting tls='webpki_and_os_roots'.
• Upgraded dependencies to newer library versions. This also includes the latest webpki-roots crate providing updated TLS CA certificate roots.
• Various example code and documentation improvements.

Bug fixes

• Fixed a bug where timestamp columns could not accept values before Jan 1st 1970 UTC.
• TCP connections now enable SO_KEEPALIVE: This should ensure that connections don't drop after a period of inactivity.

Supported Python Versions

• Now supports Python 3.12
• Drops support for Python 3.7

Full Changelog: v1.1.0...v1.2.0

What's Changed

Features

• High-performance ingestion of Pandas dataframes into QuestDB via ILP. We now support most Pandas column types. The logic is implemented in native code and is orders of magnitude faster than iterating the dataframe in Python and calling the Buffer.row() or Sender.row() methods: The Buffer can be written from Pandas at hundreds of MiB/s per CPU core. The new dataframe() method continues working with the auto_flush feature. See API documentation and examples for the new dataframe() method available on both the Sender and Buffer classes.

• New TimestampNanos.now() and TimestampMicros.now() methods. These are the new recommended way of getting the current timestamp.

• The Python GIL is now released during calls to Sender.flush() and when auto_flush is triggered. This should improve throughput when using the Sender from multiple threads.

Errata

• In previous releases the documentation for the from_datetime() methods of the TimestampNanos and TimestampMicros types recommended calling datetime.datetime.utcnow() to get the current timestamp. This is incorrect as it will (confusingly) return object with the local timezone instead of UTC. This documentation has been corrected and now recommends calling datetime.datetime.now(tz=datetime.timezone.utc) or (more efficiently) the new TimestampNanos.now() and TimestampMicros.now() methods.

Full Changelog: v1.0.2...v1.1.0

• Support for Python 3.11.
• Updated to version 2.1.1 of the c-questdb-client library:
  • Setting SO_REUSEADDR on outbound socket. This is helpful to users with large number of connections who previously ran out of outbound network ports.

• Fixed a major bug where Python int and float types were handled with 32-bit instead of 64-bit precision. This caused certain int values to be rejected and other float values to be rounded incorrectly. Closes #13.
• As a matter of convenience, the Buffer.row method can now take None column values. This has the same semantics as skipping the column altogether. Closes #3.
• Fixed a minor bug where an auto-flush error caused a second clean-up error. Closes #4.