Oh, nice find, so it seems that entities rotation is borked :)

Looking at this, if I change the number of bits passed to AngleQuantized() here to 1, I can correct the rotation, however it feels like I'm simply trimming away any useful data, and not actually resolving the issue.

Even with this change, the server is not updating the client for rotations of other character models. That is, spinning in a circle doesn't affect your model on another players screen.

What you can experiment with is forcing the rotation values to the ones we get from client:

void World::physicsStep(Entity *e,uint32_t msec)
{
    if(glm::length2(e->inp_state.pos_delta)) {
        // todo: take into account time between updates
        glm::mat3 za = static_cast<glm::mat3>(e->inp_state.direction); // quat to mat4x4 conversion
/// add the line below
        e->qrot = e->inp_state.direction;
/// add the line above
        e->pos += ((za*e->inp_state.pos_delta)*float(msec))/50.0f;
    }
}

If that does something interesting, but still wrong You might consider sending the same values the client sends to us by replacing the:

e->qrot = e->inp_state.direction;

with

e->qrot.x = e->inp_state.camera_pyr.x;
e->qrot.y = e->inp_state.camera_pyr.y;
e->qrot.z = e->inp_state.camera_pyr.z;

This is getting close, but we need to discard x,z, as we really only care about he horizontal plane (otherwise the entity will tilt up or down as you look around)

First to correct the spawn direction, I changed ent->qrot below in EntityStorage.cpp. This corrects the spawn direction for me. I'm not sure why I have to rotate -15deg, but it's the only way to straighten the characters out.

void EntityManager::InsertPlayer(Entity *ent)
{
    m_map_entities[m_last_ent++] = ent;
    ent->m_idx = m_last_ent-1;
    ent->pos = glm::vec3(128.0,16,-198); //-60.5;
    ent->qrot= glm::angleAxis(glm::radians(-15.0f), glm::vec3(0.0f, 1.0f, 0.0f)); // changed to use angleAxis
    m_entlist.push_back(ent);
}

I added the following to EntitiesResponse instead of World::physicsStep, as it updates immediately upon spawning, and continues to update as you move. The effect is not perfect, rotation only works 180deg, and rotates the opposite direction. I'm sure this is because I'm misunderstanding the math in qrot.

void EntitiesResponse::sendServerPhysicsPositions(BitStream &bs) const
{
    Entity * target = m_client->char_entity();
    bool full_update = true;
    bool has_control_id = true;

    bs.StoreBits(1,full_update);
    if( !full_update )
        bs.StoreBits(1,has_control_id);
#ifdef LOG_
    fprintf(stderr,"Phys: send %d ",target->m_input_ack);
#endif
    if( full_update || has_control_id)
        bs.StoreBits(16,target->m_input_ack); //target->m_input_ack
    if(full_update)
    {
        for(int i=0; i<3; ++i)
            bs.StoreFloat(target->pos[i]); // server position
        // Add this below:
        target->qrot = target->inp_state.direction; // Attempt to address issue #94 and update facing of entities
        // Add this above?
        NetStructure::storeFloatConditional(bs,0.0f); // PYR rotation ?
        NetStructure::storeFloatConditional(bs,0.0f);
        NetStructure::storeFloatConditional(bs,0.0f);
    }
}

Any ideas on why it's rotating in reverse? Why it'll only work 180deg?

Also, I tried to play with NetStructure::storeFloatConditional(bs,0.0f); but changing these values crashed the client for me.

Ok, the fact that you had to rotate by 15 degrees tells me something wonky is going on, maybe the client expects a PYR angles, so we'll have to convert quaternion to euler angles.
Straight from wikipedia ( although i'm not 100% the rotation order is preserved :/ )

static void toEulerAngle(const glm::quat& q, float& roll, float& pitch, float& yaw)
{
	// roll (x-axis rotation)
	float sinr = +2.0 * (q.w * q.x + q.y * q.z);
	float cosr = +1.0 - 2.0 * (q.x * q.x + q.y * q.y);
	roll = atan2(sinr, cosr);

	// pitch (y-axis rotation)
	float sinp = +2.0 * (q.w * q.y - q.z * q.x);
        if (fabs(sinp) >= 1)
            pitch = copysign(M_PI / 2, sinp); // use 90 degrees if out of range
        else
	    pitch = asin(sinp);

	// yaw (z-axis rotation)
	float siny = +2.0 * (q.w * q.z + q.x * q.y);
	float cosy = +1.0 - 2.0 * (q.y * q.y + q.z * q.z);  
	yaw = atan2(siny, cosy);
}

On my side I can look over the client and check the logic on that side.

Ok, looking over the client side, i'm pretty sure it's using PYR during transmission, so converting from quaternion to PYR is a must :/

Let me try static glm::quat QuaternionFromYawPitchRoll(const glm::vec3 &pyr) from InputState.

Please try the latest commit, it might have fixed this, if it did, feel free to close this issue :)

Also, You have my heartfelt thanks for investigating and working on this

Of course. I'll let you know my results.

This is much closer. Characters spawn with correct orientation, but still do not update their direction during movement.

Yup, we have to do some float twiddling to fix this:

void World::physicsStep(Entity *e,uint32_t msec)
{
    if(glm::length2(e->inp_state.pos_delta)) {
        // todo: take into account time between updates
        glm::mat3 za = static_cast<glm::mat3>(e->inp_state.direction); // quat to mat4x4 conversion
        glm::vec3 pyr;
        toEulerAngle(e->inp_state.direction, pyr);
        pyr.y = 0;
        pyr.z = 0;
        toQuat(pyr,e->qrot);
        e->pos += ((za*e->inp_state.pos_delta)*float(msec))/50.0f;
    }
}

I'm starting to think we should store all rotations in PYR format, and convert to quaternions/transform matrices only as needed :/

This would make sense, as most mobs and players will default to an upright position. In fact, if we're not doing any real rotational math on entities, then I don't see any reason to use quats. PYR is going to be more intuitive.

I'm assuming that flying and jumping animations would not be affected.

Tried this, but had no effect?

void World::physicsStep(Entity *e,uint32_t msec)
{
    if(glm::length2(e->inp_state.pos_delta)) {
        // todo: take into account time between updates
        glm::mat3 za = static_cast<glm::mat3>(e->inp_state.direction); // quat to mat4x4 conversion
        glm::vec3 pyr;
        toEulerAngle(e->qrot, e->inp_state.direction.x, e->inp_state.direction.y, e->inp_state.direction.z);
        pyr.y = 0;
        pyr.z = 0;
        e->qrot = QuaternionFromYawPitchRoll(pyr);
        e->pos += ((za*e->inp_state.pos_delta)*float(msec))/50.0f;
    }
}

Obviously copying the toEulerAngle and QuaternionFromYawPitchRoll functions from their respective sources.

Working on this from another angle (bad pun, sorry). I'm getting an error with this line that I don't understand.

/Segs/Projects/CoX/Servers/MapServer/Events/EntitiesResponse.cpp:363: error: invalid initialization of reference of type ‘const vec3& {aka const glm::tvec3<float, (glm::precision)0>&}’ from expression of type ‘glm::quat {aka glm::tquat<float, (glm::precision)0>}’
         target->qrot = QuaternionFromYawPitchRoll(target->inp_state.direction); // Attempt to address issue #94 and update facing of entities
                                                   ~~~~~~~~~~~~~~~~~~^~~~~~~~~

I don't understand. I think target->inp_state.direction is the same type, no? This is probably something completely obvious, but I'm missing it.

        target->qrot = QuaternionFromYawPitchRoll(target->inp_state.direction); // Attempt to address issue #94 and update facing of entities

inp_state.direction is a quaternion not a pitch-yaw-roll triple , it's converted from the pyr here:

InputStateStorage &InputStateStorage::operator =(const InputStateStorage &other)
{
///...
    if(update_needed) {
        direction = glm::angleAxis(camera_pyr[0], glm::vec3(1, 0, 0)) *
                    glm::angleAxis(camera_pyr[1], glm::vec3(0,-1, 0)) *
                    glm::angleAxis(camera_pyr[2], glm::vec3(0, 0, 1))
                ;
        //direction = glm::quat(camera_pyr[0], osg::X_AXIS, camera_pyr[1], -osg::Y_AXIS, 0, osg::Z_AXIS);
        //direction = QuaternionFromYawPitchRoll(camera_pyr);
    }
    return *this;

Oh weird. Then wouldn't target->qrot = target->inp_state.direction; just work?

When trying that before it only worked 180deg, and the rotation was reverse direction from input. Is it possible that inp_state.direction is storing the Quat in the wrong order?

It might be in wrong order, since there are waaay to many ways of converting Euler angles to quaternions :)
Also, are You sure target->qrot is converted back into EulerAngles before sending ?

Yeah, looks like we could just call glm::eularAngles() instead of using our own solution.

Continued testing on this.

Adding to SetOrientation in EntityUpdateCodec.cpp

printf("\nsrc.qrot: %s \n", glm::to_string(src.qrot).c_str());
    printf("src.inp_state.direction: %s \n", glm::to_string(src.inp_state.direction).c_str());
    printf("src.inp_state.camera_pyr: %s \n", glm::to_string(src.inp_state.camera_pyr).c_str());
    float pyr_angles[3];
    // this outputs as RPY
    toEulerAngle(src.qrot,pyr_angles[0],pyr_angles[1],pyr_angles[2]);
    printf("pyr_angles: %f %f %f \n", pyr_angles[0],pyr_angles[1],pyr_angles[2]);
    glm::vec3 pyr = glm::eulerAngles(src.inp_state.direction);
    printf("glm direction to pyr: %s \n", glm::to_string(pyr).c_str());
    glm::vec3 cam = src.inp_state.camera_pyr;
    printf("camera_pyr: %s \n", glm::to_string(cam).c_str());

Results of rotating in place in the output below, which reveals that our toEulerAngle() method differs from glm::eulerAngles() because they output RPY and ypr respectively. Also, it shows that qrot contains nothing really, and contains 0s when converted to a Euler Angle.

src.qrot: quat(0.000000, 0.000000, 0.000000, 1.000000) 
src.inp_state.direction: quat(0.001043, 0.940502, -0.002884, 0.339775) 
src.inp_state.camera_pyr: vec3(-0.006136, 2.448233, 0.000000) 
pyr_angles: 0.000000 0.000000 0.000000 
glm direction to pyr: vec3(-3.135462, 0.693359, 3.141590) 
camera_pyr: vec3(-0.006136, 2.448233, 0.000000) 

src.qrot: quat(0.000000, 0.000000, 0.000000, 1.000000) 
src.inp_state.direction: quat(0.001263, 0.911070, -0.002797, 0.412239) 
src.inp_state.camera_pyr: vec3(-0.006136, 2.291767, 0.000000) 
pyr_angles: 0.000000 0.000000 0.000000 
glm direction to pyr: vec3(-3.135449, 0.849825, -3.141585) 
camera_pyr: vec3(-0.006136, 2.291767, 0.000000) 

This would actually make sense since you begin facing south (presumably 0,0,0 orientation), but performing a translation on that does not produce desired results.

float a = glm::angle(src.inp_state.direction);
    printf("glm a: %f \n", a);

    glm::quat newrot = glm::rotate(src.qrot,a,pyr);
    printf("glm rotate: %s \n", glm::to_string(newrot).c_str());

Results in the following, and produces the undesired effect of rotating the entity in all kinds of weird directions.

src.qrot: quat(0.000000, 0.000000, 0.000000, 1.000000) 
src.inp_state.direction: quat(0.000122, 0.998937, -0.003064, 0.046003) 
src.inp_state.camera_pyr: vec3(-0.006136, 3.049554, 0.000000) 
pyr_angles: 0.000000 0.000000 0.000000 
glm direction to pyr: vec3(-3.135457, 0.092039, -3.141554) 
camera_pyr: vec3(-0.006136, 3.049554, 0.000000) 
glm a: 3.049554 
glm rotate: quat(-0.705520, 0.020710, -0.706892, 0.046003) 

src.qrot: quat(0.000000, 0.000000, 0.000000, 1.000000) 
src.inp_state.direction: quat(0.000299, 0.995028, -0.003057, 0.099543) 
src.inp_state.camera_pyr: vec3(-0.006136, 2.942175, 0.000000) 
pyr_angles: 0.000000 0.000000 0.000000 
glm direction to pyr: vec3(-3.135448, 0.199417, -3.141579) 
camera_pyr: vec3(-0.006136, 2.942175, 0.000000) 
glm a: 2.942176 
glm rotate: quat(-0.702199, 0.044661, -0.703572, 0.099543) 

src.qrot: quat(0.000000, 0.000000, 0.000000, 1.000000) 
src.inp_state.direction: quat(0.000532, 0.984476, -0.003025, 0.175493) 
src.inp_state.camera_pyr: vec3(-0.006136, 2.788777, 0.000000) 
pyr_angles: 0.000000 0.000000 0.000000 
glm direction to pyr: vec3(-3.135445, 0.352815, -3.141578) 
camera_pyr: vec3(-0.006136, 2.788777, 0.000000) 
glm a: 2.788779 
glm rotate: quat(-0.693266, 0.078010, -0.694622, 0.175493) 

All of this makes me wonder though, if the client is already handling all of this logic on it's end to show you what your character is doing when you move (which works as expected), then why doesn't it pass this info back directly to the server?

Is it possible that the client is already telling the server "I'm turning this way" and we can simply store those packets and pass them back to other clients? Isn't this how ParagonChat works over XMPP?

How can I see what the client sends the server when I'm pressing Q and E to rotate?

I'll take a closer look at this after I return home ( should be 2 days since the date of this post, I'm at code::dive conference right now )

Fun! Enjoy your trip!

Leaving myself a note:

o is the orientation. This is pitch, yaw, roll, in radians. Note that Paragon Chat, like COH, applies the transformations in the order Yaw, Pitch, Roll despite calling it a "PYR" and the numbers being in PYR order. PC rounds to the nearest hundredth of a radian.

Am I understanding this correctly?
src.inp_state.direction = quat representing the orientation of your character model, deprived from camera_pyr.
src.inp_state.camera_pyr = I think this is actually ypr. Representing camera direction
src.qrot = presumably the quat representing the rotation of your character model.

Does the client send the server a packet saying simply "this is the direction I'm facing"?

The reason I ask this is because if SetOrientation derives it's value from pyr_camera. So panning the camera around your character without rotating your model would result in your model rotating on other clients.

note: i'm at work atm, can look into orientation stuff closer on the weekend

As for panning: yup, and that's the use-case for first-person third-person camera switching ( in first person camera orientation should influence entity orientation, not so much in 3rd person )

No problem at all. I know you're busy and traveling. Thank you so much for helping me understand this.

Changed branch to https://github.com/broxen/Segs/tree/interp-v2

As you can see progress is limited. Velocity and PosUpdates need to be properly calculated.

With interp turned on, and sending filler interp data

Without interp turned on, but modifying velocity calcs

Updated branch address: https://github.com/broxen/Segs/tree/movement

Some useful articles:
http://www.gabrielgambetta.com/client-side-prediction-server-reconciliation.html
https://stackoverflow.com/questions/1448997/do-good-multiplayer-mmo-clientserver-games-use-latency-within-movement-calcula

Moving to v7 as this needs a lot more work and some testing.

Per our discord conversation
After flying around a while and going idle for a few minutes players are moved above/below the map.
It sounds like the server is updating the client with your true position.

When I move forward with mouseview facing downward, the other client sees me move downward into the ground. With mouseview facing upward, the other client sees me moving upward. The more extreme the angle of mouseview relative to the horizontal plane, the move vertical movement is produced.
When I jump, the other client sees me move upward a fixed amount, and then stay at that vertical height. Multiple jumps look like flying up vertically to the other client.

Both clients did a /stuck and the one moved toward Flint.
It seems the client thinks it can move faster than the servers current calculations.

The server is not computing jump properly, it only thinks you are going up and not down.

The server thinks you are always flying, even if you clip into something.
Here I did /stuck and moved forward while facing toward the sky.

After an afk, found blue floating where brown sees him,
which is issue #789 Random vertical teleportation.

Ahhh my old pal Movement... Closing this out, as SEGS Engine will correct these issues.