Hi All, I am using RealityKit along with ARKit and Swift UI to develop an app where I am augmenting a usdz model of a complex geometry like that of a car. I have some other usdz files with a simple plane geometry having the material properties embedded within them which also i am loading as model entities. I want to traverse through my car usdz file such that i can pick the material from simple usdz file and apply it to the car as car paint. To do this i know the name of the mesh holding the car paint as well as the name of the material applied. I have tried to traverse through the usdz files using both RealityKit and SceneKit but I am not successful to reach to the lowest mesh and copy the material properties to it. With RealityKit, I have tried to get the instance data using modelEntity as follows :- "sourceModel?.model?.mesh.contents.instances". But this returns instance id, model name and transform only. Any help will be highly appreciated. Thank You

I used other software to export usdz files, hoping to further adjust the PBR and other parameters in the model in Reality Composer Pro. Because usdz is a whole, I cannot use the mouse to select a specific model in usdz on the interface. I have to find the models I want to modify one by one in the list on the left. This method of operation is too inefficient. Is there a better way? Or is there a way to disassemble the usdz file into numerous sub-models and texture material files, so that I can select it with the mouse on the interface in Reality Composer Pro and then modify the PBR, which would be much more efficient.

Is it possible to create a roomplan with the texture of a room plan's wall - or some way to combine ObjectCapture results with RoomPlan results?

Hello, I have rendered an usdz File using sceneKit's .write() method on the displayed scene. Once I load it on another RealityKit's ARView using the .nonAR mode of the camera, I am trying to use the view's raycast(from:,allowing:,alignment:) method, to get the coordinates on the model. I have applied the collisionComponents when loading the model using the .generateCollisionShapes() function to be able to interact with the modelEntity. However, the raycast result returns nothing. Is there something I am missing for it to work? Thanks!

We have requirement adding usdz file to UIView and showing the it’s content and archive the data and save to file. When user open the file, we need to unarchive that usdz content and binding with UIView and showing it to user. Initially, we created SCNScene object passing usdz file url like below. do { usdzScene = try SCNScene(url: usdzUrl) } catch let error as NSError { print(error) } Since SCNScene support to NSSecureCoding protocol , we directly archive that object and save it file and load back it from file and created SCNScene object using NSKeyedUnarchiver process. But for some files, we realised high memory consumption while archiving the SCNScene object using below line. func encode(with coder: NSCoder) { coder.encode(self.scnScene, forKey: "scnScene") } File referene link : toy_drummer_idle.usdz When we analyse apple documentation (check discussion section) , it said, scn file extension is the fastest format for processing than the usdz. So we used SCNSecne write to feature for creating scn file from given usdz file. After that, When we do the archive SCNScene object that was created by sun file url, the archive process is more faster and it will not take high memory as well. It is really faster previous case now. But unfortunately, SCNScene write method will take lot of time for this conversion and memory meter is also going high and it will be caused to app crash as well. I check the output file size as well. The given usdz file size is 18MB and generated scn file size is 483 MB. But SCNScene archive process is so fast. Please, analyse this case and please, provide some guideline how we can optimise this behaviour. I really appreciate your feedback. Full Code: import UIKit import SceneKit class ViewController: UIViewController { var scnView: SCNView? var usdzScene: SCNScene? var scnScene: SCNScene? lazy var exportButton: UIButton = { let btn = UIButton(type: UIButton.ButtonType.system) btn.tag = 1 btn.backgroundColor = UIColor.blue btn.addTarget(self, action: #selector(buttonPressed(_:)), for: .touchUpInside) btn.setTitle("USDZ to SCN", for: .normal) btn.setTitleColor(.white, for: .normal) btn.layer.borderColor = UIColor.gray.cgColor btn.titleLabel?.font = .systemFont(ofSize: 20) btn.translatesAutoresizingMaskIntoConstraints = false return btn }() func deleteTempDirectory(directoryName: String) { let tempDirectoryUrl = URL(fileURLWithPath: NSTemporaryDirectory()) let tempDirectory = tempDirectoryUrl.appendingPathComponent(directoryName, isDirectory: true) if FileManager.default.fileExists(atPath: URL(string: tempDirectory.absoluteString)!.path) { do{ try FileManager.default.removeItem(at: tempDirectory) } catch let error as NSError { print(error) } } } func createTempDirectory(directoryName: String) -> URL? { let tempDirectoryUrl = URL(fileURLWithPath: NSTemporaryDirectory()) let toBeCreatedDirectoryUrl = tempDirectoryUrl.appendingPathComponent(directoryName, isDirectory: true) if !FileManager.default.fileExists(atPath: URL(string: toBeCreatedDirectoryUrl.absoluteString)!.path) { do{ try FileManager.default.createDirectory(at: toBeCreatedDirectoryUrl, withIntermediateDirectories: true, attributes: nil) } catch let error as NSError { print(error) return nil } } return toBeCreatedDirectoryUrl } @IBAction func buttonPressed(_ sender: UIButton){ let scnFolderName = "SCN" let scnFileName = "3D" deleteTempDirectory(directoryName: scnFolderName) guard let scnDirectoryUrl = createTempDirectory(directoryName: scnFolderName) else {return} let scnFileUrl = scnDirectoryUrl.appendingPathComponent(scnFileName).appendingPathExtension("scn") guard let usdzScene else {return} let result = usdzScene.write(to: scnFileUrl, options: nil, delegate: nil, progressHandler: nil) if (result) { print("exporting process is success.") } else { print("exporting process is failed.") } } override func viewDidLoad() { super.viewDidLoad() let usdzUrl: URL? = Bundle.main.url(forResource: "toy_drummer_idle", withExtension: "usdz") guard let usdzUrl else {return} do { usdzScene = try SCNScene(url: usdzUrl) } catch let error as NSError { print(error) } guard let usdzScene else {return} scnView = SCNView(frame: .zero) guard let scnView else {return} scnView.translatesAutoresizingMaskIntoConstraints = false self.view.addSubview(scnView) self.view.addSubview(exportButton) NSLayoutConstraint.activate([ scnView.leadingAnchor.constraint(equalTo: view.safeAreaLayoutGuide.leadingAnchor), scnView.trailingAnchor.constraint(equalTo: view.safeAreaLayoutGuide.trailingAnchor), scnView.topAnchor.constraint(equalTo: view.safeAreaLayoutGuide.topAnchor), scnView.bottomAnchor.constraint(equalTo: view.safeAreaLayoutGuide.bottomAnchor, constant: -30), exportButton.widthAnchor.constraint(equalToConstant: 200), exportButton.heightAnchor.constraint(equalToConstant: 40), exportButton.centerXAnchor.constraint(equalTo: view.safeAreaLayoutGuide.centerXAnchor), exportButton.bottomAnchor.constraint(equalTo: view.safeAreaLayoutGuide.bottomAnchor), ]) DispatchQueue.main.asyncAfter(deadline: .now() + 0.01) {[weak self] in guard let self else {return} loadModel(scene: usdzScene) } } func loadModel(scene: SCNScene){ guard let scnView else {return} scnView.autoenablesDefaultLighting = true scnView.scene = scene scnView.allowsCameraControl = true } } ![]("https://developer.apple.com/forums/content/attachment/5ff0197b-7e0b-4efe-b295-cc205c73dc02" "title=Screenshot 2024-06-20 at 12.23.40.png;width=1712;height=1013")

I've been trying to get the drag gesture up and running so I can move my 3D model around in my immersive space, but for some reason I am not able to move it around. The model shows up in my visionOS 1.0 Simulator, but I can't seem to get it to move around. Would love some help with this and some resources too that would be helpful. Here's a snippet of my Reality View code import SwiftUI import RealityKit import RealityKitContent struct GearRealityView: View { static var modelEntity = Entity() var body: some View { RealityView { content in if let model = try? await Entity(named: "LandingGear", in: realityKitContentBundle) { GearRealityView.modelEntity = model content.add(model) } }.gesture( DragGesture() .targetedToEntity(GearRealityView.modelEntity) .onChanged({ value in GearRealityView.modelEntity.position = value.convert(value.location3D, from: .local, to: GearRealityView.modelEntity.parent!) }) ) } }

I wanted to try the new Object Tracking. So I captured an object and uploaded it in Create ML in an Object Tracking project. I added my used file and set the viewing angle to "upright". Afterwards, I startet the training. After some time (around 2%), it stops and I receive following error message: "There isn't enough space. at "1921.blob". I tried it multiple times.

Here is a sample asset demoing the problem: https://cloudzeta.com/zeta/public-demo/session/lx9tkmenrcj4o5ad/quicklook.usdz Assets like this used to work well in visionOS 1, but now it's missing textures for some reason: https://www.youtube.com/watch?v=2TOMnkGvi8I

Hi everyone, I'm developing a visionOS app using SwiftUI and RealityKit. I'm facing a challenge with accessing the children of a USDZ model. Scenario: I can successfully load and display a USDZ model in my RealityView. When I import the model into Reality Composer Pro, I can access and manipulate its individual parts (children) using the scene hierarchy. However, if I directly load the USDZ model from the Navigation tab in my project, I cannot seem to access the children programmatically. Question: Is there a way to access and manipulate the children of a USDZ model loaded directly from the Navigation tab in SwiftUI for visionOS? Additional Information: I've explored using Entity(named: "childName", in: realityKitContentBundle), but this only works for the main entity. I'm open to alternative approaches if directly accessing children isn't feasible. Thanks in advance for any insights or suggestions! Best, Siddharth [Edited by Moderator]

What is the reason the hand-tracking joints have these axes? I'm trying to create a virtual hands model and that's a mess.

I am trying to store usdz files with SwiftData for now. I am converting usdz to data, then storing it with SwiftData My model import Foundation import SwiftData import SwiftUI @Model class Item { var name: String @Attribute(.externalStorage) var usdz: Data? = nil var id: String init(name: String, usdz: Data? = nil) { self.id = UUID().uuidString self.name = name self.usdz = usdz } } My function to convert usdz to data. I am currently a local usdz just to test if it is going to work. func usdzData() -> Data? { do { guard let usdzURL = Bundle.main.url(forResource: "tv_retro", withExtension: "usdz") else { fatalError("Unable to find USDZ file in the bundle.") } let usdzData = try Data(contentsOf: usdzURL) return usdzData } catch { print("Error loading USDZ file: \(error)") } return nil } Loading the items @Query private var items: [Item] ... var body: some View { ... ForEach(items) { item in HStack { Model3D(?????) { model in model .resizable() .scaledToFit() } placeholder: { ProgressView() } } } ... } How can I load the Model3D? I have tried: Model3D(data: item.usdz) Gives me the errors: Cannot convert value of type '[Item]' to expected argument type 'Binding<C>' Generic parameter 'C' could not be inferred Both errors are giving in the ForEach. I am able to print the content inside item: ForEach(items) { item in HStack { Text("\(item.name)") Text("\(item.usdz)") } } This above works fine for me. The item.usdz prints something like Optional(10954341 bytes) I would like to know 2 things: Is this the correct way to save usdz files into SwiftData? Or should I use FileManager? If so, how should I do that? Also how can I get the usdz from the storage (SwiftData) to my code and use it into Model3D?

How is it possible to add a schema for ar to a usd file using the python tools (or any other way). Following the instructions in: https://developer.apple.com/documentation/arkit/arkit_in_ios/usdz_schemas_for_ar/actions_and_triggers/preliminary_behavior The steps are to have the following declaration: class Preliminary_Behavior "Preliminary_Behavior" ( inherits = </Typed> ) and a usd file #usda 1.0 def Preliminary_Behavior "TapAndFlip" { rel triggers = [ <Tap> ] rel actions = [ <Entry> ] def Preliminary_Trigger "Tap" ( inherits = </TapGestureTrigger> ) { rel affectedObjects = [ </Cube> ] } def Preliminary_Action "Entry" ( inherits = </GroupAction> ) { uniform token type = "parallel" rel actions = [ <Flip> ] } def Preliminary_Action "Flip" ( inherits = </EmphasizeAction> ) { rel affectedObjects = [ </Cube> ] uniform token motionType = "flip" } } def Cube "Cube" { } How do these parts fit together? I saved the usda file, but it didn't have any interactions. Obviously, I have to add that declaration, but how do I do this? is this all in an AR Xcode project? Or can I do this with python tools (I would prefer something very lightweight).

Hi all, I am generating some USDZ files that will be used in quicklook and be accessible with the Vision Pro. I was wondering are there any examples with USDZ files with actions? like the ability to change a state of assesses by tapping on them? I know this works with .reality files, but I would like to use Python to create some automatically generated USDZ files that allow some interaction. I'm currently stuck! So an example of the capabilities would be great - or pointing to some code that has done this in python. Thanks!

I created a prototype app with Reality Composer on an iPad. Now I would like to import the project to a Mac for further development using Xcode/Swift. How can I do this? I am able to export a .reality or .usdz file. If I open the .reality file in Xcode I just get the playback app/scene but it does not appear to be a project that I can edit.

I am fairly new to 3D model rendering and do not know where to start. I am trying to, ideally with ARKit & RealityKit or SceneKit, do a scan of an environment. This includes: Applying realistic textures to the model. Being able to save it as a .usdz file (to be able to open it within the App itself) Once it is save do post-processing measurements within the model. I would prefer to accomplish this feature by using a mesh, instead of the pointCloud that is used in the sample project of apple. Would this be doable using Apple's APIs and on a mobile device or would it be necessary to use a third party program? I have managed to create a USDZ file using SceneKit's .scene.write(to:,delegate:) method. However the saved file is a "single object" and it is not possible to use raycasting to do post-processing measurements in the model.

I wanted to show a progress of a certain part of the game using an entity that looks like a "pie chart", basically cylinder with a cut-out. And as progress is changed (0-100) the entity would be fuller. Is there a way to create this kind of model entity? I know there are ways to animated entities, warp them between meshes, but I was wondering if somebody knows how to achieve it in a simplest way possible? Maybe some kind of custom shader that would just change how the material is rendered? I do not need its physics body, just to show it. I know how to do it in UIKit and classic 2d UI Apple frameworks but here working with model entities it gets a bit tricky for me. Here is example of how it would look, examples are in 2d but you can imagine it being 3d cylinders with a cut-out. Thank you!

I am trying to implement a way to rotate a 3D model around its y axis, but this doesn't seem to work. What am I missing? The scene only contains one model entity. @State private var rotateBy:Double = 0.0 RealityView { content in do { let entity = try await Entity.init(named: "VinylScene", in: realityKitContentBundle) entity.scale = SIMD3<Float>(repeating: 0.6) content.add(entity) } catch { ProgressView() } } .gesture( DragGesture(minimumDistance: 0.0) .targetedToAnyEntity() .onChanged { value in let location3d = value.convert(value.location3D, from: .local, to: .scene) let startLocation = value.convert(value.startLocation3D, from: .local, to: .scene) let delta = location3d - startLocation rotateBy = Double(atan(delta.x * 200)) } )

Hello. I have a model of a CD record and box, and I would like to change the artwork of it via an external image URL. My 3D knowledge is limited, but what I can say is that the RealityView contains the USDZ of the record, which in turn contains multiple materials: ArtBack, ArtFront, PlasticBox, CD. How do I target an artwork material and change it to another image? Here is the code so far. RealityView { content in do { let entity = try await Entity.init(named: "VinylScene", in: realityKitContentBundle) entity.scale = SIMD3<Float>(repeating: 0.6) content.add(entity) } catch { ProgressView() } }

Hi. Can anyone who's into cosmology give hints as to how I might depict and animate dark matter for the VisionPro?

I've got a couple 2D PNG assets that I want to add to a scene made of a couple other udsz files in RCP (picture adding a couple 2D videogame characters to a simple 3D diorama). When I try to drag the PNGs to the workspace or the file tree…nothing happens. I found a walkthrough on Medium (called "Importing and Exporting Personalized Objects for Augmented Reality: Reality Composer and SwiftUI" for those curious as I can't link to Medium posts here) that makes it look like users could do this with simple drag-and-drop. The Medium post is from June 2023, and in the screenshots RCP visually looks a lot more like Reality Composer on iPad, so I'm assuming it's changed a lot since then? Is there still a way to do this? I've tried adding the 2D elements to a scene with Blenders "import images as planes," but I'm getting weird halos around them and was hoping RCP could make the process a bit easier/cleaner.