-
Notifications
You must be signed in to change notification settings - Fork 144
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for "unknown" bits. #188
Conversation
This allows bitflags to be robustly used in cases where bit flags are defined outside the program and are subject to extension after the program has been compiled. In such cases, the new function `from_bits_unknown()` can create a bitflags structure where some of the stored bits have no pre-defined meaning to the program; nevertheless, the program can store and manipulate them successfully rather than mis-behaving.
Hi @drmikehenry 👋 This looks the same as #174 and I'm still not convinced it's an appropriate addition to This seems like a real concern for the |
I'm sorry for the duplication of pull requests. @jabedude and I discussed an early version of this idea, but due to a miscommunication I didn't realize that Josh had submitted a pull request on the topic. I read through the open issues but I now see that pull requests don't show up in the list of issues. The bitflags crate provides a nice abstraction over an integer that holds flag values. I agree that only well-defined flag values should be permitted into the integer underlying a bitflags structure. When the flag definitions are internal to the program, it's possible to define this set of valid flags at compile time, and bitflags currently handles that situation well. But in the case of a foreign function interface (FFI) like the nix crate provides, the set of valid flags can't be nailed down at compiled time because it's determined by an authority outside the program itself, where the set of valid flags may be extended at any time in the future. With this view of the meaning of valid flags, it seems natural to me that the set of valid flags is divided into those that are known at compiled time and those that aren't (or, alternatively, those which have been named at compiled time and those which are unnamed or anonymous). If we assume that only valid flags will be present in the underlying integer, then the Having a completely separate function (badly named It may be that a different name for Adding bits that are unknown/unnamed/anonymous (any better names?) seems like a natural extension to bitflags, and one that fixes a significant drawback for crates like nix that can't authoritatively define the set of valid flags at compile time. The downside is that for other use cases where the set of valid flags is completely determined at compile time, it's possible for someone to misuse Thanks for considering this issue, and again I apologize for not noticing the earlier pull request. |
Thanks for the explanation @drmikehenry! Ok, I can appreciate your perspective, so I think we could explore something like Just as a stream-of-consciousness, in order to determine whether or not this function should be Do you have a need to distinguish the bits you 'know' from the bits you don't? |
- Change `from_bits_unknown()` to the `unsafe` function `from_bits_unchecked()`. - Eliminate unnecessary concept of "known" and "unknown" bits.
I like the word "unchecked". I think it gives the right impression about what's going on. To me, the big win BitFlags provides is the type safety you get between two different For example, the C library function int fcntl(int fd, int cmd, ... /* arg */ ); Depending on the value of
File descriptor flags and file status flags have different definitions and interpretations; here are two such definitions taken from the #define FD_CLOEXEC 1 // a file descriptor flag for use with F_GETFD/F_SETFD
#define O_WRONLY 1 // a file status flag for use with F_GETFL/F_SETFL When trying to set fcntl(fd, F_SETFD, fcntl(fd, F_GETFD) | FD_CLOEXEC); // intended
fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) | FD_CLOEXEC); // mistake! The second line uses fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) | 1);
fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) | O_WRONLY); // actual call The mistaken call has the effect of setting the file's write-only bit, which will typically succeed and might easily be a no-op, while failing to set the close-on-exec bit, causing a subtle leak of a file descriptor into an Use of In answer to your question, in general I don't have any use case for querying a I've updated the pull request along the above lines. Let me know what you think. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't forget to update the CHANGELOG, too.
Thanks all for your patience here! I'm just dropping a line here to let you know that I'll give this the proper review it deserves next week 🙂 I'm just buried in some other things at the moment. |
@KodrAus Take your time - I've got a work-around in the meantime, and it's worth making sure you are satisfied with the approach. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, thanks for your patience @jabedude and @drmikehenry!
This looks good to me.
bors r+ |
188: Add support for "unknown" bits. r=KodrAus a=drmikehenry The nix crate uses bitflags heavily, as it provides a type-safe way of dealing with the many different types of flags in the system. In most cases, the bit flag definitions are specified by the system, and the nix project then provides bitflags-based definitions for these flags. This leads to a robustness problem as described below. The bitflags structure will not allow "unknown" bit flag values to be stored. All possible flag values must be known at compile time. Once an executable is compiled, there is no way for it to gracefully handle system extensions that define new flag values. Because many of these values are read from system calls, the program can receive bit flag values it did not know about at compile time, causing unnecessary errors for the previously correct program. The nix issue linked below shows an example where this behavior of bitflags leads to a less robust program that can't handle system extensions made after the program was compiled: nix-rust/nix#1102 For a hypothetical case, imagine a set of system-defined flags that can be read from the system and written back after modification. To set `SOME_FLAG`, the following C code might be used: ```c int flags = get_system_flags(); flags = flags | SOME_FLAG; set_system_flags(flags); ``` This correctly handles any possible combinations of bits the operating system might supply, even if those bit flags weren't defined when the C program was compiled. Using bitflags, it's not possible to get the same level of future-proofing; either `from_bits()` is used, causing an error when unknown bits are provided by the system, or `from_bits_truncate()` is used, throwing away these unknown bits and silently corrupting the state of the flags. When the authoritative source of flag definitions is within the program itself, it makes sense to prohibit unknown flag values in bitflags structures; but when the bit flag definitions evolve in a backward-compatible fashion but independently from the program, the only way to robustly handle post-compile-time bit flag extensions is to allow unknown bits into a bitflags structure. This pull request's purpose is to add a third method of converting from bits into a bitflags structure that accepts arbitrary bits without error or truncation: ```rust pub const fn from_bits_unknown(bits: $T) -> $BitFlags { $BitFlags { bits } } ``` With this extension, a bitflags structure can represent any possible set of bits the system deems valid. Once created, the structure retains its type safety benefits and allows modification using the subset of bit flag definitions known to the program at compile time. Unless the programmer uses `from_bits_unknown()`, there is now way for unknown bits to creep into the structure; for cases where it's important to allow for the unknown bits, the use of this function stands as a clear marker to the reader. Co-authored-by: Michael Henry <drmikehenry@drmikehenry.com>
Build failed
|
I don't really know how bors works, and I don't have access to the report results for the failure above. I'm assuming therefore that there's nothing I should be doing here, but please let me know if the above failure indicates I need to change the pull request somehow. |
bors retry |
188: Add support for "unknown" bits. r=KodrAus a=drmikehenry The nix crate uses bitflags heavily, as it provides a type-safe way of dealing with the many different types of flags in the system. In most cases, the bit flag definitions are specified by the system, and the nix project then provides bitflags-based definitions for these flags. This leads to a robustness problem as described below. The bitflags structure will not allow "unknown" bit flag values to be stored. All possible flag values must be known at compile time. Once an executable is compiled, there is no way for it to gracefully handle system extensions that define new flag values. Because many of these values are read from system calls, the program can receive bit flag values it did not know about at compile time, causing unnecessary errors for the previously correct program. The nix issue linked below shows an example where this behavior of bitflags leads to a less robust program that can't handle system extensions made after the program was compiled: nix-rust/nix#1102 For a hypothetical case, imagine a set of system-defined flags that can be read from the system and written back after modification. To set `SOME_FLAG`, the following C code might be used: ```c int flags = get_system_flags(); flags = flags | SOME_FLAG; set_system_flags(flags); ``` This correctly handles any possible combinations of bits the operating system might supply, even if those bit flags weren't defined when the C program was compiled. Using bitflags, it's not possible to get the same level of future-proofing; either `from_bits()` is used, causing an error when unknown bits are provided by the system, or `from_bits_truncate()` is used, throwing away these unknown bits and silently corrupting the state of the flags. When the authoritative source of flag definitions is within the program itself, it makes sense to prohibit unknown flag values in bitflags structures; but when the bit flag definitions evolve in a backward-compatible fashion but independently from the program, the only way to robustly handle post-compile-time bit flag extensions is to allow unknown bits into a bitflags structure. This pull request's purpose is to add a third method of converting from bits into a bitflags structure that accepts arbitrary bits without error or truncation: ```rust pub const fn from_bits_unknown(bits: $T) -> $BitFlags { $BitFlags { bits } } ``` With this extension, a bitflags structure can represent any possible set of bits the system deems valid. Once created, the structure retains its type safety benefits and allows modification using the subset of bit flag definitions known to the program at compile time. Unless the programmer uses `from_bits_unknown()`, there is now way for unknown bits to creep into the structure; for cases where it's important to allow for the unknown bits, the use of this function stands as a clear marker to the reader. Co-authored-by: Michael Henry <drmikehenry@drmikehenry.com>
Build failed
|
bors retry |
188: Add support for "unknown" bits. r=KodrAus a=drmikehenry The nix crate uses bitflags heavily, as it provides a type-safe way of dealing with the many different types of flags in the system. In most cases, the bit flag definitions are specified by the system, and the nix project then provides bitflags-based definitions for these flags. This leads to a robustness problem as described below. The bitflags structure will not allow "unknown" bit flag values to be stored. All possible flag values must be known at compile time. Once an executable is compiled, there is no way for it to gracefully handle system extensions that define new flag values. Because many of these values are read from system calls, the program can receive bit flag values it did not know about at compile time, causing unnecessary errors for the previously correct program. The nix issue linked below shows an example where this behavior of bitflags leads to a less robust program that can't handle system extensions made after the program was compiled: nix-rust/nix#1102 For a hypothetical case, imagine a set of system-defined flags that can be read from the system and written back after modification. To set `SOME_FLAG`, the following C code might be used: ```c int flags = get_system_flags(); flags = flags | SOME_FLAG; set_system_flags(flags); ``` This correctly handles any possible combinations of bits the operating system might supply, even if those bit flags weren't defined when the C program was compiled. Using bitflags, it's not possible to get the same level of future-proofing; either `from_bits()` is used, causing an error when unknown bits are provided by the system, or `from_bits_truncate()` is used, throwing away these unknown bits and silently corrupting the state of the flags. When the authoritative source of flag definitions is within the program itself, it makes sense to prohibit unknown flag values in bitflags structures; but when the bit flag definitions evolve in a backward-compatible fashion but independently from the program, the only way to robustly handle post-compile-time bit flag extensions is to allow unknown bits into a bitflags structure. This pull request's purpose is to add a third method of converting from bits into a bitflags structure that accepts arbitrary bits without error or truncation: ```rust pub const fn from_bits_unknown(bits: $T) -> $BitFlags { $BitFlags { bits } } ``` With this extension, a bitflags structure can represent any possible set of bits the system deems valid. Once created, the structure retains its type safety benefits and allows modification using the subset of bit flag definitions known to the program at compile time. Unless the programmer uses `from_bits_unknown()`, there is now way for unknown bits to creep into the structure; for cases where it's important to allow for the unknown bits, the use of this function stands as a clear marker to the reader. Co-authored-by: Michael Henry <drmikehenry@drmikehenry.com>
Alrighty, I'll give this one more try and if bors is still unhappy I'll merge manually :) |
Build failed
|
31: Bump bitflags from 1.1.0 to 1.2.0 r=fkarg a=dependabot-preview[bot] Bumps [bitflags](https://github.com/bitflags/bitflags) from 1.1.0 to 1.2.0. <details> <summary>Release notes</summary> *Sourced from [bitflags's releases](https://github.com/bitflags/bitflags/releases).* > ## 1.2.0 > - Fix typo: {Lower, Upper}Exp - {Lower, Upper}Hex ([#183](https://github-redirect.dependabot.com/bitflags/bitflags/issues/183)) > > - Add support for "unknown" bits ([#188](https://github-redirect.dependabot.com/bitflags/bitflags/issues/188)) > > [#183](https://github-redirect.dependabot.com/bitflags/bitflags/issues/183): [bitflags/bitflags#183](https://github-redirect.dependabot.com/rust-lang-nursery/bitflags/pull/183) > [#188](https://github-redirect.dependabot.com/bitflags/bitflags/issues/188): [bitflags/bitflags#188](https://github-redirect.dependabot.com/rust-lang-nursery/bitflags/pull/188) </details> <details> <summary>Changelog</summary> *Sourced from [bitflags's changelog](https://github.com/bitflags/bitflags/blob/master/CHANGELOG.md).* > # 1.2.0 > > - Fix typo: {Lower, Upper}Exp - {Lower, Upper}Hex ([#183](https://github-redirect.dependabot.com/bitflags/bitflags/issues/183)) > > - Add support for "unknown" bits ([#188](https://github-redirect.dependabot.com/bitflags/bitflags/issues/188)) > > [#183](https://github-redirect.dependabot.com/bitflags/bitflags/issues/183): [bitflags/bitflags#183](https://github-redirect.dependabot.com/rust-lang-nursery/bitflags/pull/183) > [#188](https://github-redirect.dependabot.com/bitflags/bitflags/issues/188): [bitflags/bitflags#188](https://github-redirect.dependabot.com/rust-lang-nursery/bitflags/pull/188) </details> <details> <summary>Commits</summary> - [`405d92d`](bitflags/bitflags@405d92d) Merge pull request [#190](https://github-redirect.dependabot.com/bitflags/bitflags/issues/190) from KodrAus/cargo/1.2.0 - [`3aca2ad`](bitflags/bitflags@3aca2ad) don't run integration test suite on 1.20.0 - [`cde9cb6`](bitflags/bitflags@cde9cb6) prepare for 1.2.0 release - [`4880a76`](bitflags/bitflags@4880a76) Merge pull request [#188](https://github-redirect.dependabot.com/bitflags/bitflags/issues/188) from drmikehenry/unknown - [`80fcb6a`](bitflags/bitflags@80fcb6a) Adjust pull request based on feedback: - [`8b9676e`](bitflags/bitflags@8b9676e) Add support for "unknown" bits. - [`de30308`](bitflags/bitflags@de30308) Merge [#186](https://github-redirect.dependabot.com/bitflags/bitflags/issues/186) - [`4e762d0`](bitflags/bitflags@4e762d0) fix up unused item warning - [`fc5f206`](bitflags/bitflags@fc5f206) fix up failing test and ensure it runs in CI - [`8a10bdc`](bitflags/bitflags@8a10bdc) Merge [#183](https://github-redirect.dependabot.com/bitflags/bitflags/issues/183) - Additional commits viewable in [compare view](bitflags/bitflags@1.1.0...1.2.0) </details> <br /> [![Dependabot compatibility score](https://api.dependabot.com/badges/compatibility_score?dependency-name=bitflags&package-manager=cargo&previous-version=1.1.0&new-version=1.2.0)](https://dependabot.com/compatibility-score.html?dependency-name=bitflags&package-manager=cargo&previous-version=1.1.0&new-version=1.2.0) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language - `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com): - Update frequency (including time of day and day of week) - Pull request limits (per update run and/or open at any time) - Out-of-range updates (receive only lockfile updates, if desired) - Security updates (receive only security updates, if desired) Finally, you can contact us by mentioning @dependabot. </details> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
@KodrAus Thanks for the merge; bitflags 1.2.0 is working fine for me :-) |
The nix crate uses bitflags heavily, as it provides a type-safe way of dealing
with the many different types of flags in the system. In most cases, the bit
flag definitions are specified by the system, and the nix project then provides
bitflags-based definitions for these flags. This leads to a robustness problem
as described below.
The bitflags structure will not allow "unknown" bit flag values to be stored.
All possible flag values must be known at compile time. Once an executable is
compiled, there is no way for it to gracefully handle system extensions that
define new flag values. Because many of these values are read from system
calls, the program can receive bit flag values it did not know about at compile
time, causing unnecessary errors for the previously correct program.
The nix issue linked below shows an example where this behavior of bitflags
leads to a less robust program that can't handle system extensions made after
the program was compiled:
nix-rust/nix#1102
For a hypothetical case, imagine a set of system-defined flags that can be read
from the system and written back after modification. To set
SOME_FLAG
, thefollowing C code might be used:
This correctly handles any possible combinations of bits the operating system
might supply, even if those bit flags weren't defined when the C program was
compiled. Using bitflags, it's not possible to get the same level of
future-proofing; either
from_bits()
is used, causing an error when unknownbits are provided by the system, or
from_bits_truncate()
is used, throwingaway these unknown bits and silently corrupting the state of the flags.
When the authoritative source of flag definitions is within the program itself,
it makes sense to prohibit unknown flag values in bitflags structures; but when
the bit flag definitions evolve in a backward-compatible fashion but
independently from the program, the only way to robustly handle
post-compile-time bit flag extensions is to allow unknown bits into a bitflags
structure.
This pull request's purpose is to add a third method of converting from bits
into a bitflags structure that accepts arbitrary bits without error or
truncation:
With this extension, a bitflags structure can represent any possible set of bits
the system deems valid. Once created, the structure retains its type safety
benefits and allows modification using the subset of bit flag definitions known
to the program at compile time. Unless the programmer uses
from_bits_unknown()
, there is now way for unknown bits to creep into thestructure; for cases where it's important to allow for the unknown bits, the use
of this function stands as a clear marker to the reader.