This website uses cookies. By clicking Accept, you consent to the use of cookies. Click Here to learn more about how we use cookies.
Accept
Reject
Can you scan QR codes with the passthrough camera?
Go to solution
sean_yang0498
Honored Guest
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-25-202204:08 AM - last edited Tuesday by 
TheLegend27
Hello, Oculus community,
I am an internship student and I got a task to figure out if it's possible to get images from the passthrough cameras on Quest 2. The purpose is to scan a QR code for creating AR experiences. I don't have any development experience with Oculus and AR, but only some experience in Unity.
I googled it and saw some results. But most of them are posted months or even years before. So I am not sure how is it now in the latest update. I also noticed there are some project showcases that used the passthrough cameras for AR experiences, but none of them used the camera for scanning QR codes.
Is there a solution to my question? I really appreciate any help.
Solved!Go to Solution.
Labels:
- Labels:
- Meta Quest 2
- Camera
- passthrough
- QR code
- scanning
6Kudos
- All forum topics
- Previous Topic
- Next Topic
1 ACCEPTED SOLUTION
Accepted Solutions
Go to solution
kojack
MVP
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-29-202208:05 PM
Sadly this is not possible.
The Oculus SDK doesn't provide any direct access to the cameras, there is no way to get pass through video. (This is for privacy reasons, imagine an unethical VR dev getting a feed of your cameras)
The passthrough API that you would have seen (with cool games like Hauntify) doesn't actually give the passthrough feed to the game. It stays in the runtime. The game can control where passthrough or game gfx are rendered, but it can't see what was rendered (it's not sent to Unity as a texture).
I want fiducial marker (like QR code) recognition for large area tracking (to correct for drift and sync multiple headsets to the same tracking space) but the only way to do it is stick a webcam to the Quest and use that (software support is going to be tricky though).
Author: Oculus Monitor,Auto Oculus Touch,Forum Dark Mode, Phantom Touch Remover,X-Plane Fixer
Hardware: Threadripper 1950x, MSI Gaming Trio 2080TI, Asrock X399 Taich
Headsets: Wrap 1200VR, DK1, DK2, CV1, Rift-S, GearVR, Go, Quest, Quest 2, Reverb G2
View solution in original post
6Kudos
15 REPLIES 15
Go to solution
james_griffin_ibm
Honored Guest
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-25-202211:58 AM
I am also looking for a solution to this in Unity3d. This would help enable enterprise cloud app authentication.
1Kudo
Go to solution
kojack
MVP
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-29-202208:05 PM
Sadly this is not possible.
The Oculus SDK doesn't provide any direct access to the cameras, there is no way to get pass through video. (This is for privacy reasons, imagine an unethical VR dev getting a feed of your cameras)
The passthrough API that you would have seen (with cool games like Hauntify) doesn't actually give the passthrough feed to the game. It stays in the runtime. The game can control where passthrough or game gfx are rendered, but it can't see what was rendered (it's not sent to Unity as a texture).
I want fiducial marker (like QR code) recognition for large area tracking (to correct for drift and sync multiple headsets to the same tracking space) but the only way to do it is stick a webcam to the Quest and use that (software support is going to be tricky though).
Author: Oculus Monitor,Auto Oculus Touch,Forum Dark Mode, Phantom Touch Remover,X-Plane Fixer
Hardware: Threadripper 1950x, MSI Gaming Trio 2080TI, Asrock X399 Taich
Headsets: Wrap 1200VR, DK1, DK2, CV1, Rift-S, GearVR, Go, Quest, Quest 2, Reverb G2
6Kudos
Go to solution
sean_yang0498
Honored Guest
In response to kojack
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-02-202201:32 AM
Alright, thank you anyway for giving me a clear answer to this!
0Kudos
Go to solution
SurrealGames
Protege
In response to kojack
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-08-202209:17 AM
the privacy answer makes no sense, and certainly could be a setting that let's the user decide. you're disabling functionality that would actually enable Hololens/Magic Leap type apps for no useful reason.
6Kudos
Go to solution
Chetpih
Honored Guest
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-08-202305:17 AM - edited 02-08-202305:18 AM
Hi! As an intern exploring the Oculus community, I understand the challenges of getting up to speed quickly on new technology. For your task, explore OCR (optical character recognition) technology and see if any solutions would allow you to scan QR codes using the passthrough cameras on Quest 2. I've heard promising things about a company called Smart Engines, which specializes in this area, but I recommend researching and checking out their latest updates to see if they have what you're looking for. Good luck with your internship project!
0Kudos
Go to solution
kojack
MVP
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-13-202305:41 AM
As I said above, this is NOT possible. No access to the images from the passthrough camera is provided by the SDK.
Author: Oculus Monitor,Auto Oculus Touch,Forum Dark Mode, Phantom Touch Remover,X-Plane Fixer
Hardware: Threadripper 1950x, MSI Gaming Trio 2080TI, Asrock X399 Taich
Headsets: Wrap 1200VR, DK1, DK2, CV1, Rift-S, GearVR, Go, Quest, Quest 2, Reverb G2
1Kudo
Go to solution
meredithtbeasley94
Honored Guest
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-14-202302:18 AM - edited 03-16-202302:31 AM
It's possible to access the passthrough cameras on Quest 2 and use them for AR experiences.
As for scanning QR codes with the passthrough cameras, it's definitely worth looking into.
Also, consider integrating a document verification and scanning solution like https://www.idanalyzer.com/products/prime-id-scanner.html into your AR experience. It offers advanced features like ID verification, biometric facial recognition, and document scanning that could be a great fit for your project.
0Kudos
Go to solution
kojack
MVP
In response to meredithtbeasley94
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-14-202304:28 AM
To read a QR code, the app needs access to the image data coming from the cameras.
The passthrough API does not provide app access to the image data coming from the cameras.
What the passthrough API does is allow you to add a passthrough layer to the rendering stack, then adjust the transparency of the layers to control where the passthrough appears. You can render VR stuff on top of the passthrough (like the guardian setup). You can render passthrough on top of VR (like a window into the real world). What you can't do is actually access any pixel of the passthrough to do things like QR code processing or OCR. All 3rd party AR apps for Quests (1,2,Pro) just use these features. Meta themselves can access the data to do things like detecting certain keyboards, but 3rd party developers can't do that.
Author: Oculus Monitor,Auto Oculus Touch,Forum Dark Mode, Phantom Touch Remover,X-Plane Fixer
Hardware: Threadripper 1950x, MSI Gaming Trio 2080TI, Asrock X399 Taich
Headsets: Wrap 1200VR, DK1, DK2, CV1, Rift-S, GearVR, Go, Quest, Quest 2, Reverb G2
0Kudos
- «Previous
- 1
- 2
- Next»
Related Content
- Improve pass through quality on Meta Quest 2 with AI upscaling in Ideas
- Meta Quest build v64 release notes in Announcements
- Quest 3 Tracking Lost after Depth sensor damage. in Get Help
- Green Screen Passthrough Mode (Enhanced Passthrough) in Ideas
- Quest 3 Passthrough - New Strategy to Avoid Deformation in Ideas
'); $(this).append(divContainer); $(divContainer).fadeIn(); var userId = $(this).attr('href').replace(/.*\/user-id\//gi,''); var windowWidth = $(window).width(); var left = $(this).offset().left; var cardWidth = divContainer.outerWidth(); if ((left + cardWidth) > (windowWidth - 25)) { var adjustment = (left + cardWidth) - (windowWidth + 25) + 50; divContainer.css('left', (-1 * adjustment) + 'px'); } $.ajax({ url: '/plugins/custom/facebook/fboculus/theme-lib.profile-card?tid=-1467189201015938772', type: 'post', dataType: 'html', data: {"userId": userId}, beforeSend: function() {}, success: function(data) { $('.info-container', divContainer).append(data); }, error: function() { $('.info-container', divContainer).append('
'); }, complete: function() { $('.spinner', divContainer).remove(); } }); }); $('body').on('click', '.user-profile-card', function(evt) { if (!$(evt.target).hasClass('profile-link')) { evt.preventDefault(); } evt.stopPropagation(); }); }); })(LITHIUM.jQuery);LITHIUM.AjaxSupport({"ajaxOptionsParam":{"event":"LITHIUM:lightboxRenderComponent","parameters":{"componentParams":"{\n \"surveyType\" : {\n \"value\" : \"communityexperience\",\n \"class\" : \"java.lang.String\"\n },\n \"surveyId\" : {\n \"value\" : \"3\",\n \"class\" : \"java.lang.Integer\"\n },\n \"triggerSelector\" : {\n \"value\" : \"#valueSurveyLauncher\",\n \"class\" : \"lithium.util.css.CssSelector\"\n }\n}","componentId":"valuesurveys.widget.survey-prompt-dialog"},"trackableEvent":false},"tokenId":"ajax","elementSelector":"#valueSurveyLauncher","action":"lightboxRenderComponent","feedbackSelector":false,"url":"https://communityforums.atmeta.com/t5/forums/v5/forumtopicpage.liabase.basebody.valuesurveylauncher.valuesurveylauncher:lightboxrendercomponent?t:ac=board-id/General/thread-id/383757","ajaxErrorEventName":"LITHIUM:ajaxError","token":"nJx58gLFx_pPCuwXGakGx7qKOesJJ7kGooYl4QNqjWM."});LITHIUM.Dialog.options['-1681927448'] = {"contentContext":"valuesurveys.widget.survey-prompt-dialog","dialogOptions":{"minHeight":399,"draggable":false,"maxHeight":800,"resizable":false,"autoOpen":false,"width":610,"minWidth":610,"dialogClass":"lia-content lia-panel-dialog lia-panel-dialog-modal-simple lia-panel-dialog-modal-valuesurvey","position":["center","center"],"modal":true,"maxWidth":610,"ariaLabel":"Feedback for community"},"contentType":"ajax"};LITHIUM.Dialog({ "closeImageIconURL" : "https://communityforums.atmeta.com/skins/images/9A34BC80128479B71497882AE7DA4B36/theme_hermes/images/button_dialog_close.svg", "activecastFullscreen" : false, "dialogTitleHeadingLevel" : "2", "dropdownMenuSelector" : ".lia-menu-navigation-wrapper", "accessibility" : true, "triggerSelector" : ".lia-panel-dialog-trigger-event-click", "ajaxEvent" : "LITHIUM:lightboxRenderComponent", "focusOnDialogTriggerWhenClosed" : false, "closeEvent" : "LITHIUM:lightboxCloseEvent", "defaultAriaLabel" : "", "dropdownMenuOpenerSelector" : ".lia-js-menu-opener", "buttonDialogCloseAlt" : "Close", "dialogContentCssClass" : "lia-panel-dialog-content", "triggerEvent" : "click", "dialogKey" : "dialogKey"});LITHIUM.ValueSurveyLauncher({"detectPopUpCSS":".lia-dialog-open","dialogLinkSelector":"#valueSurveyLauncher","launchDelay":184246});LITHIUM.PartialRenderProxy({"limuirsComponentRenderedEvent":"LITHIUM:limuirsComponentRendered","relayEvent":"LITHIUM:partialRenderProxyRelay","listenerEvent":"LITHIUM:partialRenderProxy"});LITHIUM.AjaxSupport({"ajaxOptionsParam":{"event":"LITHIUM:partialRenderProxyRelay","parameters":{"javascript.ignore_combine_and_minify":"true"}},"tokenId":"ajax","elementSelector":document,"action":"partialRenderProxyRelay","feedbackSelector":false,"url":"https://communityforums.atmeta.com/t5/forums/v5/forumtopicpage.liabase.basebody.partialrenderproxy:partialrenderproxyrelay?t:ac=board-id/General/thread-id/383757","ajaxErrorEventName":"LITHIUM:ajaxError","token":"FBxiu7bTujcL37vHcL5yZIt6a3Q4R3lqKl8jgS0u95E."});LITHIUM.Auth.API_URL = "/t5/util/authcheckpage";LITHIUM.Auth.LOGIN_URL_TMPL = "/plugins/common/feature/oauth2sso_v2/sso_login_redirect?referer=https%3A%2F%2FREPLACE_TEXT";LITHIUM.Auth.KEEP_ALIVE_URL = "/t5/status/blankpage?keepalive";LITHIUM.Auth.KEEP_ALIVE_TIME = 300000;LITHIUM.Auth.CHECK_SESSION_TOKEN = '2qukQ0SbkmnnyDXOz6kela7r45IbAtAfXIKA09XS9aI.';LITHIUM.AjaxSupport.useTickets = false;LITHIUM.Cache.CustomEvent.set([{"elementId":"link_12","stopTriggerEvent":false,"fireEvent":"LITHIUM:labelSelected","triggerEvent":"click","eventContext":{"uid":1717,"selectedLabel":"meta quest 2","title":"Meta Quest 2"}},{"elementId":"link_68","stopTriggerEvent":false,"fireEvent":"LITHIUM:changePage","triggerEvent":"click","eventContext":{"parameters":{"page":2}}},{"elementId":"link_69","stopTriggerEvent":false,"fireEvent":"LITHIUM:changePage","triggerEvent":"click","eventContext":{"parameters":{"pageNavigationDirection":"next","page":2}}}]);LITHIUM.Loader.runJsAttached();});// -->
