Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is Proptest abandoned? #268

Closed
russelldb opened this issue May 26, 2022 · 37 comments
Closed

Is Proptest abandoned? #268

russelldb opened this issue May 26, 2022 · 37 comments

Comments

@russelldb
Copy link

There are a lot of open PRs, the project's own tests no longer pass (since #258) (TBH I can't even get the tests to compile)

Not to be one of those entitled downstream users, but can @AltSysrq please let us know the state of the project, or if someone should fork/takeover as the canonical crate?

Many thanks in advance

@nzeh
Copy link

nzeh commented Jul 28, 2022

There's an updated fork here: https://github.com/nzeh/proptest. I don't really have the intention of taking over maintainership of this crate. I'll submit a pull request shortly and would prefer that my changes get incorporated into the main repo.

My fork addresses two issues, alas via (hopefully minor) breaking changes:

  • The tests in proptest and proptest-derive all pass again. This came at the cost of removing half-bounded ranges as strategies for f32 and f64. Those were the failing tests for proptest. After looking at how the rand crate generates floats in a range and how proptest tries to use that generator, I concluded that there is no way to make the tests pass. Moreover, the tests are fairly representative of what should be a common use of half-bounded float ranges as strategies, so they would fail also in real-world use and simply shouldn't be supported.

  • In my opinion, the current shrinking strategy for values produced by prop_flat_map is almost always a poor choice. It first tries to shrink the inner value and only when this fails, shrinks the outer value on which the inner value depends. I ran into this when trying to generate a graph with a number of vertices and a set of vectors that store attributes, one per vertex. The most natural way to do this is to generate the number of vertices and then prop_flat_map this to a strategy that generates a graph with that number of vertices and a set of vectors, each of this length. Shrinking is completely ineffective here because it will try tons of possibilities of messing with the vector entries and with the edge set of the graph before trying to shrink the number of vertices. As a result, shrinking will either take hours or will fail to produce a meaningfully shrunk input, depending on the bound on the number of shrinks.

    I reimplemented the shrinking strategy for FlattenValueTree so it shrinks the outer value first and only when that fails, refines the result by shrinking the inner value. Unfortunately, given the API in the Strategy trait, I didn't see a way to do this without requiring that the strategy produced by the closure given to prop_flat_map implement Clone, a requirement that wasn't there before. This may break some custom strategies, but I expect it to break less code than trying to mess with the Strategy trait itself and, at least in my use cases, seems like a small price to pay for getting a useful shrinking strategy from prop_flat_map. I expect there to be a discussion whether there's a better way to do this, and I'm happy to engage in that.

@rex-remind101
Copy link
Collaborator

Unless either of you have been contacted in private, it seems at this point like the library has been abandoned. @nzeh should your fork be designated the new official one? Should a new organization be formed to host the project with new owners?

@nzeh
Copy link

nzeh commented Sep 8, 2022

There has been no response. I'm reluctant to have my fork designated the official one because my ability to dedicate time to this project is really unpredictable. There will likely be long stretches when I wouldn't be able to put any time aside for this, so it would likely end up in the same situation as the current official repo and at least seem abandoned. I'd be more than happy to be part of a team that co-maintains official fork.

@russelldb
Copy link
Author

We at Ditto (https://github.com/getditto) have been leaning pretty heavely on Proptest. We have a fork that we are using. Happy to discuss with management about us taking ownership. Spoke with @tzemanovic about this a little too.

What do people think of that?

@nzeh
Copy link

nzeh commented Sep 8, 2022

I'm good with that (not that I feel a single commit gives me a major say in that anyway :) ).

@AltSysrq
Copy link
Collaborator

AltSysrq commented Sep 8, 2022

Hi all, apologies for my long silence here. To answer the title question: Yes, pretty much. I had gotten completely burnt out after I started burning almost every weekend dealing with questions/features.

At this point I'd be happy to help transfer ownership if a suitable org were set up (or if Ditto itself wants to take it). My only reservation with transferring to an individual is the potential for abuse (no offence to any of you — I'm just overly cautious) but if they have already established trust in the community that would be fine too.

@russelldb
Copy link
Author

Good to hear from you! I've spoken to colleagues at Ditto, and we'll take it on, if that is acceptable to you and the wider community

@rex-remind101
Copy link
Collaborator

rex-remind101 commented Sep 8, 2022

I'd prefer if an org that was associated with the community at large was built. I think that's preferable to the longevity of a library since it can then be grounded in a more diverse subset of the community, also given how important of a role this library plays across rust. Of course the challenging part is establishing that initial set of owners.

@cameron1024
Copy link
Member

FWIW my employer also has a fork at https://github.com/input-output-hk/proptest that is lightly maintained (only a few minor features added like const generics).

We heavily rely on proptest, and have got huge value from it. I'd be happy to dedicate some personal time to helping out.

As for who the ownership should go to, I largely agree that a community organization would be ideal, but I'd also be happy with Ditto, especially if that means people are getting paid for the maintenance 😁

@rex-remind101
Copy link
Collaborator

@AltSysrq would you be able to start a new organization, add interested parties as owners, and transfer ownership to this organization?

@matusf
Copy link

matusf commented Sep 19, 2022

Just an idea. Maybe a @rust-fuzz would be a right home for this project? It consists of established and trusted members of Rust community and property-based testing and fuzzing are closely related. However, I am not part of the organization So we would need to talk to them to find out whether they would be interested. I would not like to drop responsibilities on someone. Maybe at the begining they could have the deployment rights and act as an authority.

@AltSysrq
Copy link
Collaborator

I'd be fine with either, but in the interest of getting this done, I'm inclined to take the path we can go unilaterally; i.e., the new org. Specifically:

  1. People who want a stake in the new org volunteer here in this thread.
  2. In a few days I'll set up the new org and transfer ownership.

@rex-remind101
Copy link
Collaborator

I'm open to taking a stake in ownership, however, my main contributions being code review and issue review. There are some additions I've wanted to make to proptest but I have yet to have the capacity to commit time to them.

If we are to create a new org rather than relying on rust-fuzz, we'll need to negotiate how we conduct business (e.g. how many different reviewers' approval before merge; do we meet at any cadence or only use asynchronous comms; what are the channels; etc.). However, the new org can also decide to try and transfer ownership to them after all. Just some bits I feel are necessary to raise, I'm on board with this plan assuming others are too 👍

@matthew-russo
Copy link
Member

I'm interested in taking a stake in ownership, pending some conversations and approval from my employer.

@cameron1024
Copy link
Member

I'd also be happy to take some stake in ownership. Agree with the above comments about needing some sort of process in place. We probably also need to think about how we reconcile everyone's respective forks.

If it's useful, I can ask my employer if there's capacity for work time to be dedicated to maintenance 👍

@AltSysrq
Copy link
Collaborator

@matthew-russo @cameron1024 Please do have those conversations if you can. (And sorry that I lost track of this again.)

@matthew-russo
Copy link
Member

I'm good to help maintain. It would be in my free time though, just to set expectations clearly.

@cameron1024
Copy link
Member

I'm in the same boat. I usually have a fair amount of time on weekends.

As for reconciling forks, I'm not sure what the best strategy is. Our company's fork only had a few trivial changes, not all of which should be included IMO. I'd be happy to open a PR to copy over the useful stuff (mainly just Arbitrary impls for const generics and a few other types)

@seanmajorpayne
Copy link

@russelldb Hey thanks for offering to take ownership of this project. Did your organization end up taking over? My company would also like to rely on this and if you haven't taken ownership or have some blocker we are up for it. If there is a separate repo you've set up, let us know.

@russelldb
Copy link
Author

@russelldb Hey thanks for offering to take ownership of this project. Did your organization end up taking over? My company would also like to rely on this and if you haven't taken ownership or have some blocker we are up for it. If there is a separate repo you've set up, let us know.

The subsequent discussions in this thread seemed to be pretty keen not to have some commercial entity "take ownership", so no, we did not. HOWEVER, it remains the case that there is no canonical continuation descendent owner of this repo. Ditto depend very much on an unmerged PR that adds stateful testing. And so we have to have a fork that we maintain. Currently I guess it is essentially "vendored". We would love for there to be a decision and progress made on a new home.

@AltSysrq
Copy link
Collaborator

I've set up the new org and invited @rex-remind101, @matthew-russo, @cameron1024 as members. I'll add to the crates.io permissions once the invitations are accepted. @russelldb Did you also want to be a member?

@rex-remind101
Copy link
Collaborator

Thanks for setting this up @AltSysrq! Just accepted the invite. Once @russelldb makes a decision I suggest we figure out comms channel and then from there discuss code review and, as others have said, how/in what order we'll review and merge what sounds like quite a few forks into our main repo.

@russelldb
Copy link
Author

I've set up the new org and invited @rex-remind101, @matthew-russo, @cameron1024 as members. I'll add to the crates.io permissions once the invitations are accepted. @russelldb Did you also want to be a member?

Yes please! Many thanks.

@AltSysrq
Copy link
Collaborator

Huh, I thought I'd be notified when the invitations were accepted.

Anyway, I've set up a publish team in proptest-rs and granted publish rights to those in it.

@rex-remind101
Copy link
Collaborator

rex-remind101 commented Nov 2, 2022

Hi yall, how would everyone prefer to communicate so we can start merging in people's forks? I could start a slack org and temporarily provide an invite link here as a suggestion?

@matthew-russo
Copy link
Member

Slack works with me.

@russelldb
Copy link
Author

russelldb commented Nov 3, 2022

Likewise, Slack is good for me too.

@cameron1024
Copy link
Member

Same here

@rex-remind101
Copy link
Collaborator

Ok, I will setup something tonight, thanks everyone 👍

@rex-remind101
Copy link
Collaborator

@matthew-russo
Copy link
Member

@AltSysrq whenever you get the chance can you update the team you've already created to allow write access to the repository? Right now we can't merge any PRs.

@AltSysrq
Copy link
Collaborator

AltSysrq commented Nov 6, 2022

Whoops, should be fixed now.

@seanmajorpayne
Copy link

@matthew-russo @russelldb @AltSysrq

Would you all be willing to have me join the group of maintainers as well? I'd like to contribute :)

@rex-remind101
Copy link
Collaborator

@seanmajorpayne I can help you join if you are still interested. Here's a link to slack, I can help you there https://join.slack.com/t/proptest-rs/shared_invite/zt-1lk8opcx1-Ssh6G0sQjEwGm6AouCtKUQ

I'll be closing this topic now :)

@tzemanovic
Copy link
Contributor

@seanmajorpayne I can help you join if you are still interested. Here's a link to slack, I can help you there https://join.slack.com/t/proptest-rs/shared_invite/zt-1lk8opcx1-Ssh6G0sQjEwGm6AouCtKUQ

I'll be closing this topic now :)

Hey, sorry I'm late to the party! I'd be interested in joining slack and the maintenance team. Could you provide a fresh link? The last one is expired

@tzemanovic
Copy link
Contributor

@rex-remind101 bump

@matthew-russo
Copy link
Member

@tzemanovic https://join.slack.com/t/proptest-rs/shared_invite/zt-1rwigqgdf-AI4jWy6u3w27bOm5R9fN9Q

this one shouldn't be time-limited. i'll look in to adding this to our CONTRIBUTING page. In general though anyone is welcome to open issues and PRs. The slack workspace is primarily just for some discussion on what we're working on or what we want each other to look at.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants