Thank you for reporting this. I'm able to reproduce this discrepancy if I don't provide any velocity towards the wall, as you point out.

This looks to be a regression caused by #559, as reverting that change fixes this problem, but that of course reintroduces the issue that it was meant to fix, meaning ghost collisions.

I'll see what can be done to fix this while preserving that other fix.

If this helps in any way, fiddling around with the Resolver settings in the Jolt 3D settings and setting it to around 20% or lower will make it behave more consistently, but I assume it also should when left to the default value when it did in previous versions.

Sounds good. Thanks for investigating this issue :)

I am facing the exact same issue with CharacterBody3D nodes.

Minor update:

As of #860 this regression has now technically been resolved, but unfortunately this issue still remains.

From what I understand, the reason for this is because of some fairly fundamental issues with the part of move_and_slide that's on the Godot side of things, where it doesn't distinguish between the collision checks that's meant to detect its environment for things like is_on_floor, is_on_wall, etc. and the collision checks that's meant to solve the actual collision. This leads to having to make a choice between sacrificing one or the other, which is not ideal.

So I can resolve this issue to some degree by under-reporting the actual penetration that happens, but that will instead lead to an increase in ghost collisions, where the character will randomly stop for no apparent reason, so that isn't really an alternative.

In fact, this issue seems to be reproducible with Godot Physics as well, although the issue is less pronounced there, and mostly seems to happen when colliding with a ConcavePolygonShape3D.

With that I'll go ahead and tag this as external, as it's unlikely that this will ever be fixed without first addressing these fundamental issues on the Godot side of things.

I been doing quite a lot of research, including investigating the GJK based seep method collision started some time ago.

I'm still in a fight with compiling, to be honest. C++ is giving me headaches. But i hope to eventually get there.

In basic terms the approach of that PR is, in essence, ray marching. So instead of doing per shape sweep, the process consist on the following: GJK is performed against a non intersecting shape. This gives a distance which is the minimum distance between two points of the shapes, but we don't know which points. So we move this minimum distance and Check with GJK again. This cycle repeats until the body do collide when moved the minimum distance.

Now, i didn't got into code yet, but i'm very much tempted to think that the problems that PR originated in move and collide are born out of the way in which the safe and unsafe fraction are calculated.

Previously with binary search safe and unsafe where the last position it didn't collide and the last position it did.

The PR just divides the motion in 64 pieces (Magic number, problem number one) and offset the result of the motion test.

The problem for me is that, with sweep testing, we assume we arrived exactly to the other shape, so adding nothing to it makes no sense to me. You are only loosing precision that we fought hard to gain with sweeping.

The new safe fraction should result in a stable motion, which can never be if we just add 1/64 of the motion provided.

What I believe the solution is is to calculate a safe fraction that keeps the body away from the contact point exactly by the safe margin. This guarantee that the only randomness is float error and GJK precision. This would also make the first part of body test motion way less necessary. Right now, we always "overlap" at the beginning, which induces the collision resolution against all the stuff, which can be A LOT of triangles. If the safe margin is now part of the collision fraction the body should, in general terms, keep the distance necessary with any geo to not have to perform this part of the test every time, which could solve our issues with Godot crawling when moving against concave shapes.

Of course, it also prevents that all the hard work put into the sweep testing don't get wasted because it will be "unstuck" in a random direction the next frame.

Team said i'm free to tackle this as long as we don't fully break backwards compatibility. This is basically finishing the implementation Reduz started, so we can be pretty sure is a valid option for the team.

So for me that's the next step. I'll try to update on whatever i think is relevant to you, as i understand that even if we get this working in godot there may be some work to do to make sure Jolt works with that.

Honestly, i hope this works, because if its the case the tunneling is gone and the precision should be way better. When that is the case, we i can go back and tackle move and slide itself. I personally not a fan of the current character body but i do think is possible to replicate is functionality without the bugs and maybe provide a better way to handle walls and floors while being the same node for the end user.

So yeah, things seem to be moving. Sorry for not being capable of providing solutions quicker, i literally had no idea of any of this not so long ago.

Thank you @mihe for your work, really a pleasure to have you around.

EDIT: Don't kill me for more text, but i'm thinking, does thing like the shapecast node uses body test motion too? If not, i should look into that, because a GJK sweep is orders of magnitude more precise than any discrete method.

does thing like the shapecast node uses body test motion too? If not, i should look into that, because a GJK sweep is orders of magnitude more precise than any discrete method.

No, it doesn't use body_test_motion, it uses cast_motion and get_rest_info, but those essentially do exactly what step 2 and step 3 does for body_test_motion, meaning a binary search. In fact, I've been tempted several times to refactor body_test_motion on the Godot side of things to rely entirely on the existing physics queries provided by PhysicsDirectSpaceState*D, which should be doable, so that body_test_motion doesn't have to be re-implemented by each and every physics server.

Godot should definitely provide a sweep test method as part of PhysicsDirectSpaceState*D, since that's how practically every other physics engine does shape-casting (that I've seen at least). The only thing that has stopped me from making a PR to add it myself is the daunting task of adding support for it to Godot Physics, but obviously Juan's PR is a decent starting point there.

However, I don't think it's feasible to replace cast_motion (nor ShapeCast*D) with a sweep implementation without breaking backwards compatibility, since you'll need to maintain the existing interface of returning the safe and unsafe fractions, and you don't have the luxury of a guaranteed safe margin there, so producing a safe/unsafe fraction from GJK/EPA will likely be difficult, if not impossible. In fact I ran into this exact issue in my first iteration of cast_motion for Godot Jolt, which utilized Jolt's sweep test, as seen in jrouwe/JoltPhysics#450.

Thank you. I read it all and is what i expected. I went ahead and ported the Reduz PR to master, and adjusted the fractions with as much common sense as i could, at least to see if this was viable as a hack. But even if the sweep is correct, all the final collision resolution is still done the same way, without having into account the sweep. As you very well described, any benefit for the sweep itself is lost in further processing.

I could make the PR to Godot with the fraction adjusted as he told, but in deep testing i see all the issues you had, but increased by the iterative nature of move and slide. I personally don't think the PR do any benefit, at the end of the day all the normal information is gathered without it and it is only adding issues.

Any recommendation of how to proceed? I feel like i felt in a really similar trap that what you described implementing shape casting. It is of any worth to make the PR available to show that, even adjusted, it does not work?

I agree with you, sweeping should provide coherent and complete sweeping methods. I been trying to improve this at least enough to make it suitable to use in our game Liblast and help people in general, but as you know by this point, move and slide can not be properly implemented without reliable data, and we don't have a good way to get reliable data.

We also had made use of some of this non desired behavior for features, for example having to under resolve to be able to get data at all in some scenarios. What i'm trying to say is that i think even more than before than this is a puzzle than needs revision.

All i could do i did, do you think is anything more i could try and tackle? i was told to try and research this stuff for the moment the team is willing to hire someone, should my report be the one i pictured to you? is there any other path of improvement that does not meant revisiting the whole foundation? Liblast is an open source project meant to test and contribute back to Godot, but we don't gain anything doing it and i don't have any other income. If we already did everything possible from our perspective, it would be good to know so we can focus on other things and i can focus on releasing free assets until this is fixed on the future. You are the person who had worked with godot physics the most by far, so i would really appreciate an overview of the situation from your side.

PS: To make clear why this is on topic, i could now surrender with having better data and try and redo move and slide. But it would be a redo using the same unreliable tooling, so it is sure it would need to be rewritten again on the future. So i don't know to which extend we can start thinking about everything kinematic / move and slide related as Godot 5 material. Fixing this particular issue with walls would probably cause other issues.

Any recommendation of how to proceed? I feel like i felt in a really similar trap that what you described implementing shape casting. It is of any worth to make the PR available to show that, even adjusted, it does not work?

I don't have any suggestions, unfortunately.

All i could do i did, do you think is anything more i could try and tackle? i was told to try and research this stuff for the moment the team is willing to hire someone, should my report be the one i pictured to you? is there any other path of improvement that does not meant revisiting the whole foundation? Liblast is an open source project meant to test and contribute back to Godot, but we don't gain anything doing it and i don't have any other income. If we already did everything possible from our perspective, it would be good to know so we can focus on other things and i can focus on releasing free assets until this is fixed on the future. You are the person who had worked with godot physics the most by far, so i would really appreciate an overview of the situation from your side.

Again, I'm not as deep into this subject as you seem to be. Character controllers are still a blindspot of mine, and are notoriously difficult to implement, as I'm sure you're aware. While I've spent an awful amount of time debugging body_test_motion and move_and_slide, I can't claim to truly understand either well enough to say whether you've exhausted all your options or not.

So i don't know to which extend we can start thinking about everything kinematic / move and slide related as Godot 5 material.

From what I understand the aim is to not have a Godot 5 for many years, meaning any such breaking change will likely need to happen as part of Godot 4, which means that it will likely need to have strong and well-researched arguments, or be highly desired by the community.

The problem with move_and_slide is that it sort of works in its current state for basic use-cases, but seems to fall apart pretty quickly when you want more out of it, and with Godot still mostly attracting the beginner/intermediate segment of the market it strikes me as difficult to get community backing for any radical breaking changes to move_and_slide.