Oculus Rift S Support #225
Comments
|
Got my Rift S; added the device ID to rift.c with some other minor edits, but it didn't seem to work. examples/simple just spits out zeroes. diff --git a/src/drv_oculus_rift/rift.c b/src/drv_oculus_rift/rift.c
index c8bc96f..d59cfa7 100644
--- a/src/drv_oculus_rift/rift.c
+++ b/src/drv_oculus_rift/rift.c
@@ -47,7 +47,7 @@ typedef enum {
REV_DK1,
REV_DK2,
REV_CV1,
-
+ REV_RIFTS,
REV_GEARVR_GEN1
} rift_revision;
@@ -280,7 +280,7 @@ static ohmd_device* open_device(ohmd_driver* driver, ohmd_device_desc* desc)
set_coordinate_frame(priv, priv->coordinate_frame);
// Turn the screens on
- if (desc->revision == REV_CV1)
+ if (desc->revision == REV_CV1 || desc->revision == REV_RIFTS)
{
size = encode_enable_components(buf, true, true, true);
if (send_feature_report(priv, buf, size) == -1)
@@ -362,7 +362,8 @@ static ohmd_device* open_device(ohmd_driver* driver, ohmd_device_desc* desc)
ohmd_set_universal_distortion_k(&(priv->base.properties), 1.003, -1.005, 0.403, 0.599);
ohmd_set_universal_aberration_k(&(priv->base.properties), 0.985, 1.000, 1.015);
break;
- case REV_CV1:
+ case REV_CV1:
+ case REV_RIFTS:
ohmd_set_universal_distortion_k(&(priv->base.properties), 0.098, .324, -0.241, 0.819);
ohmd_set_universal_aberration_k(&(priv->base.properties), 0.9952420, 1.0, 1.0008074);
/* CV1 reports IPD, but not lens center, at least not anywhere I could find, so use the manually measured value of 0.054 */
@@ -428,6 +429,7 @@ static void get_device_list(ohmd_driver* driver, ohmd_device_list* list)
{ "Rift (DK2)", OCULUS_VR_INC_ID, 0x0021, -1, REV_DK2 },
{ "Rift (DK2)", OCULUS_VR_INC_ID, 0x2021, -1, REV_DK2 },
{ "Rift (CV1)", OCULUS_VR_INC_ID, 0x0031, 0, REV_CV1 },
+ { "Rift (RIFTS)", OCULUS_VR_INC_ID, 0x0051, 0, REV_RIFTS },
{ "GearVR (Gen1)", SAMSUNG_ELECTRONICS_CO_ID, 0xa500, 0, REV_GEARVR_GEN1 },
};
~ |
|
probably something with the package format or something like that, a wireshark dump would be really nice to get started for us, since no one has a S in our team. |
I've done this for network connections but never USB .. I assume you mean something that's dumping USB? |
|
wireshark can do usb capture as well! look for usbmon/usbpcap |
|
TIL! OK - will figure this and get something over to you next day or so. |
|
BTW - I'm on a Mac. Not sure if that makes a diff. |
|
So the Rift S has two virtual USB 3.0 Hubs which has the following devices: I did a USBPcap recording while it was working (I did through the first setup). I will upload it shortly. |
|
Ok, here is the Wireshark recording of the USB Hub. I just hope I got right since I never did that in Windows: https://www.teske.net.br/lucas/rift/RiftUsbCapture.7z I will try to nail down each device. |
|
And that's the full list of VID/PID |
|
SteamVR in windows seems to think there are 3 base stations present with the Rift S. I wonder if the tracking of ground and controllers are handled on the device and pass data to virtual base stations or if the hmd driver would need to process data from all the cameras and IMUs. Either way.... i think a guardian/calibration/setup will need to be implemented on *nix systems for the Rift S to work properly. I hope those HID Devices are 3 virtual base stations, 2 controllers, and a helmet. That would make everything easier. |
|
Yep. Thats correct @Akm0d. And also the oculus software when running in passthrough (camera view), doesnt show anything in the Mirror on the pc, which means the camera data never goes back via usb 3. I might have more time this weekend to play a bit and make some PoC code that can read everthing. |
|
Ok just as reference, the three Rift HIDs are named: 0: "Rift S Tracking Data" The other two that windows finds as HID: 0: USB-CDC Modem ( loads as /dev/ttyACM0, has debug messages from the headset) So @Akm0d I was wrong about the HID Devices. Still I don't see a lot of traffic on the "camera" device when its running in-game, I just see when the passthrough mode is enabled somehow. Maybe thats used for mirror? |
|
Also, I can't get the video to be enabled in my linux, not even with the HMD setting at nvidia config. |
|
On windows you get camera passthrough during guardian setup and when you leave the guardian area, I wonder if there is some kind of flag or command to trigger receiving that data over USB. I haven't poked around with it too much in linux... I'll see what I can find out some nights this week. |
|
@Akm0d I mean, if you enable oculus mirror, it usually doesn't show the passthrough camera in the PC side. Anwyays, figured out some stuff. If you write 0x1 to HID report 0x14 of "Rift S System State", it turns on the Camera device (it popus up as UVC on linux) and enables the controllers. If you send 0x1 to Report 0x02, it turns on the HMD. I did a small snippet that does that: Will work more soon. |
|
Ok, after few more tests I also narrowed down the RiftS initialization procedure: This will make it available in the Display Port with the AllowHMD setting in NVidia.
Still, I got no image at Rift S. The Rift S log say its running fine: About the controllers, they can be found at Interface 0x08. def ParseController(data):
reportId = data[0]
controllerId = binascii.hexlify(data[1:9])
data = data[9:]
c = " (%02d " % ord(data[0])
v = struct.unpack("<I", data[1:5])[0]
c += " %d) " % v
if ord(data[0]) != 32:
return
data = data[5:]
c += "(%03d " % ord(data[0])
o, v = struct.unpack("<HH", data[1:5])
c += " %05d %05d) " % (o, v) // Packet Counter
data = data[5:]
s = struct.unpack("<H", data[:2])
c += " %05d " % s
data = data[2:]
s = struct.unpack("<3h", data[:6])
c += " (%05d %05d %05d)" % s // Acell
data = data[6:]
c += " (%05d %05d %05d) " % struct.unpack("<3h", data[:6]) // Gyros
data = data[6:]
print "%s: %s [%s]" % (controllerId.upper(), c, toHex(data).upper()) |
|
@racerxdl Bruh. People like you are the heroes that make Linux thrive (and, in this case, MacOS gaming by extension). I know you still have to get an image from the S, but you've already done a lot in such a short time. +1 for your efforts, and for the project as a whole. Edit: Will try this script for myself since I have a Rift S, and I'll post how that goes here. |
|
@racerxdl Not sure what I did wrong, but I got a bunch of compiler errors with both scripts. Which C compiler/version did you use? (Sorry if this is a dumb question, I don't know much about the differences between compilers and versions!) Edit: I used Linux Mint as my OS, and I tested with GCC (and G++) version 7.4.0. Thought you should know since I asked you. |
|
@EdgyBama GCC 7.3, standard from ubuntu 18. gcc rifts-init.c -lhidapi-libusb -o rifts-initShould work fine in 7.4 as well. What are the errors? |
|
@racerxdl I don't remember what the errors are, but it doesn't really matter anymore, since your command compiled the code, with some warnings, and I was able to run the script properly. I'm pretty sure, since I've never compiled c or c++ using any libraries before, that I mistook the "l" for an "I" when typing my command initially. I'll post the warnings below if you really need them, but they seem like something you're already well aware of.: Edit: just for clarification, I tested both scripts. Both (appear to) work. Edit 2: Wow. I really need to clarify my results the first time. rift-s-test results: rifts-init results: It does not turn on the physical screen like you said, but the white LED lights up and the screen shows up in the NVidia settings. |
|
The warnings I get here, is just because I didn't bothered to make it the right way (since its just a test snippet). They're harmless for that case. I still need to do some more work on Rift-S stuff, sadly it looks like I can't get it to work in my laptop with the GTX980 so I will need to use my desktop that has two GTX 1060. I'm just wondering why there is no image since there is not much more payloads the headset receives from USB (most of the payloads are flash memory control to get settings from the headset itself). This week there is a holiday in Brazil so I will have more time to check it. The only sad thing is that I have no other headset to compare with, so I have no clue how one of these should work. |
|
If I remember correctly from using the Rift S on Windows 10, the headset didn't show any image until I turned on the Oculus App (from FB's Website), SteamVR will not start without it. The Oculus App, when Opened, opens the Oculus Desktop (by default), turns on the cameras, etc, etc. Not sure if me telling you this helps you out any. I'm saying there might have to be a call from the OS/OpenHMD (possibly using some Open Source VR Desktop) that is used to turn on the Headset Display, potentially using your existing payload information. Edit/Disclaimer: I know very little about how OpenHMD/Rift S Hardware Support actually works. I do have a general idea about how hardware and OS calls work, and have a pretty good understanding of how to do general troubleshooting, and that's where my hypothesis comes from. Also have fun on your vacation! |
|
The call from Oculus App is the one you just ran there. The headset display chipset is a ANX7533 (https://www.analogix.com/en/system/files/AA-004747-PB-1-ANX7533_0.pdf) with a FPGA for Camera Processing. The Oculus App also deals in a generic way regarding any supported headset (the drivers in the windows are just generic HID, so the app controls everything). So its probably something stupid that I let pass. Also just for completeness, the LCD screen is a LS055R1SX04. Funny enough, its overclocked in the Rift S (Probably by low persistence / reduced range). |
|
Gotcha. as for not having another headset to compare with, post the link to your results when you're ready and the minute I get a chance I'll test it out with my Rift S like I did before 👍 . |
|
I can't believe I didn't state my specs after posting that it worked for me: Linux Mint 19.1 Edit: Dependency issue forced me to switch to Manjaro 18. Everything else still applies. |
|
@racerxdl Have you put any work into wrapping it into a OpenHMD driver? or would you like some help with that. If so please post the latest gist you have and i can get working on that. Also keep in mind that on the CV1 you need a application opening the usb device and active on the Oculus screen before the screen enables, might be part of the issue you are having |
|
Hi @TheOnlyJoey, to be honest I haven't put too much work on it because for some reason I have some limitations on my GTX 980 which didn't happend on the GTX 1060. So I waited until I finished building my desktop to continue the work. The current status is: The Rift S shows as a secondary screen on linux with the correct resolution (2560x1440) and right refresh rate, the controllers are received on USB but the rift does not display the image even the status led showing its OK. I think I might missed some USB transactions, so I need to take a deep look on that. Besides that, I will need some help wrapiing into OpenHMD driver, mainly because: to get the Rift S running, there are two USB devices that I need to talk, and OpenHMD assumes that there is only one VID/PID/Interface wrapped. |
|
Ok so I haven't saw the #231 PR, I will check that to see how it was managed in CV1. |
So as mentioned before, this is exactly the same as the CV1, you need to move a rendered window to the device in order for it to start display anything, while covering the light field sensor that is inside the device. We also send a keepalive message, might be a good idea to wrap your test into our rift driver and see what it does. |
|
@TheOnlyJoey Apologies for the potentially stupid question, but I can't find any info on this: Is/are the light field sensor(s) on the Rift S the cameras on the front and side? If not, where is it? I want to try this for myself if I can... Edit: I know @racerxdl 's changes haven't been posted yet. I'm asking for when it's available. |
|
Yep, thanks for noticing. I just hacked my way through in the cmake file when I was doing it yesterday too and forgot to write about that, sorry |
|
I had a quite fruitless attempt at doing my own measurements of the lens distortion over the weekend. Everything I came up with was arguably worse than the CV1 distortion, using the cameras I have on hand. I've ordered a new camera, so I'll try that again in a few weeks. In the meantime, I just fixed a bug calculating the time delta for IMU fusion that was making the tracking really bad. That's working noticeably better now. |
|
I implemented controller support in https://github.com/thaytan/OpenHMD/tree/dev-rift-s this weekend. IMU rotational tracking and controls reports now work. Very occasionally I see spurious failures at startup with some of the USB traffic reading the controller configuration blocks. That probably needs some retry logic, but in general it seems to work quite well. |
This is a preliminary 3DOF driver for the Rift S headset and controllers. Rotational IMU tracking and controller controls are supported. It would also be possible to publish more from the controllers with additional OpenHMD API, such as capsense touch information for each button/trigger. Distortion parameters are taken from the CV1 and need improving, but aren't completely wrong. Partially fixes OpenHMD#225
This is a preliminary 3DOF driver for the Rift S headset and controllers. Rotational IMU tracking and controller controls are supported. It would also be possible to publish more from the controllers with additional OpenHMD API, such as capsense touch information for each button/trigger. Distortion parameters are taken from the CV1 and need improving, but aren't completely wrong. Partially fixes OpenHMD#225
|
Can't get Rift S working in SteamVR. It's looking for 2560x1440 screen but my main screen has same resolution and SteamVR choses it as correct. When I disconnect main screen SteamVR is not seeing Rift S at all but HMD appearing in xrandr. |
|
That's probably better filed as a new issue now that this one is closed and initial Rift S support merged. |
Just to clarify, is the Rift S ready to play games yet? |
|
No. You can check the openhmd website for a list of working devices and their support level. |
|
Current support is for 3DOF only (you can look around but not move). Controller buttons work, but without being able to move the controllers around it's only useful in limited situations for now. |
|
Hello! I just spent the last few days getting balls deep inside RiftS.dll on windows with IDA Pro, and I might have some useful info. I know that you guys can already access the camera streams on linux because it shows up as a standard UVC device, but I have figured out how to access the stream on Windows. The driver uses the MSMF API (Microsoft Media Foundation), which is a standard backend for webcams, except you need to set an extra flag while enumerating the devices to pick up the Rift S cameras. The easiest way to test this is to clone OpenCV, add one line of code, build it, and make a simple test program. See my issue on opencv for details. In the test app, all you need to do is to open the VideoCapture with the int main() {
VideoCapture cap;
cap.open(CV_CAP_MSMF + 0); // +0 is the device ID, but it's obviously not needed here
cap.set(CAP_PROP_FOURCC, VideoWriter::fourcc('M', 'J', 'P', 'G'));
cap.set(CAP_PROP_FRAME_WIDTH, 3200);
cap.set(CAP_PROP_FRAME_HEIGHT, 856);
cap.set(CAP_PROP_FPS, 60);
Mat mat;
while(true) {
cap >> mat;
imshow("cam", mat);
if(waitKey(10) == 27)
break;
}
return 0;
}Also some more info, probably not as useful, here is a complete list of every single HID data packet (only name, ID and size so far) that can be sent from the driver to the device. These values are scraped straight from the decompiled source code, so they are 'official'. I don't have a lot of time right now, but when I do, I will try to build detailed package descriptions about as much of them as I can. |
|
I just want to say I'm so glad for Rift S owners that they finally have hope to use the Rift S outside of Windows. Thank you to all of the people in this thread who've actually contributed to this part of OpenHMD. I don't own mine anymore because it took too much work to get set up on Linux at the time, and I was in Grad School with a bunch of other projects that were more immediately important to me personally to get working. I acknowledge that VR is critical for many people's use cases, I just had others that demanded my attention that second. I wish I knew more about how Wireshark worked at the time of when I owned my Rift S and had the time to learn more about the differences of C verisons. When I was a full-time Linux user I would have been able to be able to actually contribute had that been the case. I regret not contributing to this part of the project more when I had the opportunity. Maybe that'll happen when I get the ability to use VR again by way of Mobile Phone Headset/Wii Remote and Kinect (I do not own a Kinect currently which is keeping me from doing this right now), but I'll cross that bridge when I get to it... |
|
the Rift S has an AI on board to help with tracking not sure how you should replicate something like that or if its necessary |
The Rift S doesn't have anything "on board". The device is basically a $400 webcam that streams the video from its 5 sensors + some IMU data through the USB cable into the Oculus software, which uses some sort of proprietary SLAM algo to track the movement of the headset. There is only very minimal data processing happening on the device itself. This can definitely be replicated, and there are already lots of open source projects that do exactly that. (e.g. msckf_vio, Kimera-VIO, ORB_SLAM3, etc...). I've been working on this a few months ago, and I even made some progress with tracking, but then life got complicated and now I don't have enough time to work on it. But it is 100% doable. |
|
if you have tracking working would you be willing to make a pull request or a fork with that I have a Rift S and would love to help test that stuff |
|
Is there support for Oculus quest 2? |
|
(Sorry if this comment re-opens this issue.) Does anyone know the progress of support for the Oculus Rift S on Linux? I know that OpenHMD doesn't fully support the device yet, and I'm currently struggling to replicate my actually getting the display to work on KDE Plasma, but I was curious if any form of position tracking was close. I recently upgraded my GPU to an AMD Radeon 7900 XT, and now the headset keeps malfunctioning due to an issue in the Windows registry. (Thanks, Microsoft, for your work on this beautiful operating system.) |
|
As far as i know, nothing new :( |
|
Guys, You should change to Monado. |
Does it support Oculus Rift S, and does it work on Steam VR? |
Yes - I moved the OpenHMD Rift S driver to Monado last year, and development has been happening there instead. Monado has a SteamVR driver implementation. Controller 6dof tracking is still WIP, but head-tracking SLAM works, and hand tracking |
|
Is it possible to use Monado with SteamVR, and is it possible to play VR games like Beat Saber via Proton? And how is hand tracking supported, but controllers are not? EDIT: Just watched a video on hand tracking; that is awesome. |
A few brave souls have tried, using the hand-tracking controller emulation. They say it's not great :)
The hand tracking is an amazing piece of work - implementing a full hand tracking set of neural networks and logic from scratch. The controller tracking - I have wanted to concentrate on one implementation at a time, so focussed on controller tracking for the CV1 headset driver with the idea to port it to other headsets once it's perfect. I've come to the conclusion that it'll never be perfect though and I should just get on with it. It takes a long time, because I can only work on VR drivers in my spare time.
It is! |
|
Question(I would love to help with this stuff, but I'm still new to C and I've never dealt with driver development or reverse engineering, though I would like to get into both; and I don't know much about how VR headsets actually work): if hand tracking is possible via the cameras on the HMD tracking the hands, and the controllers already have rotation support, why not simply add a system that can track the controllers using the same system as the hand tracking? Granted it would not be perfect, because if the headset loses track of the controllers, they'd be lost period, but it's a start. I have no idea how controller tracking works on the Oculus Rift S in the first place. Since it has no cameras on it, I assume the headset either uses the above mentioned system in a way, or it uses signal triangulation. |
|
Rift-S has quite a few cameras on it. Indeed if ( in windows ) you step outside the guardian area the game view goes away and a black and white camera view appears instead. The controllers have infra-red LED's in the rings and it's these that the cameras on the headset pick up in order to determine position and rotation. |
|
@AgentLoneStar007 wrote:
Rift-S has 5 outward facing tracking cameras. If you look at infrared pics of the controller you can see the constellation of invisible infrared LEDs, which is how it knows where they are: |
|
Cool. Alright, I know this isn't the best place to ask, considering it's an issue for a Github project and all, but could someone either help me figure out Monado, or link a guide for installation and usage of it on Arch Linux? If you want a better place to talk, reply to this post. I've installed the SteamVR driver, I've tried to use the |
|
Monado (monado-git) is packaged in Arch. Install the optional basalt-monado-git and onnxruntime deps first to have SLAM and hand tracking. |
and others
I know there is no support yet (since it not even got release oficially).
But I got the pre-order for Rift S and I plan to contribute with OpenHMD to support.
So I'm just adding this issue so I do not forget about it, and if someone also have one (or will have) and want to help, it can use this issue.
The text was updated successfully, but these errors were encountered: