Is this level of ground plane flickering to be expected?

I'm Stuck
1

Include the following details (edit as applicable):

  • Issue category: Semantic Segmentation / Real-time Mapping-Depth
  • Device type & OS version: Android Google Pixel 4
  • Host machine & OS version: Windows 10
  • Issue Environment : On Device
  • Xcode version:
  • ARDK version: 2.1
  • Unity version: 2021.3.6f

Description of the issue:

I’m using the “Gameboard” coupled with its debug visualisation (Visualiser.cs). As the video shows there’s consistently a large amount of flickering of the gameboard. I tend to encounter this problem with the ARVoyage app as well running on this same phone. I’ve found a previous thread that seems to have this same problem (Detected plane comes on and off). There are times when the gameboard runs a lot more smoothly and there’s a lot less flickering, but under other situations like this it just constantly flickers. FYI I’m already using the Semantic Segmentation feature to suppress the “ground” layer, though depth manager is currently set to “Auto” (I’ve tried it on Screen Space Mesh setting as well but results seem to be similar)

Is this simply the current technical limitation of ARDK? Are there any “best practices” or setups to minimise this flickering?

2

This is depth occlusion being turned on. Sadly you cannot exclude meshes from it, so of something flat is on top of or near real world surfaces you need to turn on depth occlusion culling

3

Hi @Kang-An,

There may be a couple of things you can try to decrease the flickering effect in your scene.

First, on the ARSemanticSegmentationManager component, you can try increasing the Key Frame Frequency value to match the application framerate at a 1:1 ratio. I found that setting this value to 30 – as well as setting the Key Frame Frequency value of the ARDepthManager to 30 – helped ensure the ground channel suppression was occurring as often as possible, and flickering in my test scene was reduced to only occurring when viewing the ground at a direct 90° angle.

Secondly, you may want to add the natural_ground and artificial_ground channels to the Depth Suppression Channels list. The ground channel you added should include both of these channels, but you may want to try experimenting with a different combination of suppression channels to see if the issue is reduced in your environment.

Additionally, I will file a feature request for the gameboard for positioning the tiles at a predetermined height above the detected plane, rather than at plane-level. Being able to position the gameboard tiles at, say, 0.1 unit above the ground plane may help prevent the z-fighting that you’re seeing from occurring in a future ARDK release.

4

Hi @rob_link thanks for the info, I’ll test out your suggestions. Just FYI I think the documentation is outdated, there’s no longer an option for “natural_ground” under semantic segmentation labels. I get an error, had to change it to “grass”. I’m pretty sure I’m on ARDK 2.1

image

5

Thank you for reporting this.

As of ARDK 2.1.0, there does seem to be a discrepancy between the available channels listed in the documentation and the ChannelName enum in the MockSemanticLabel class, and I’ve filed a report to let the team know.

6

The team has confirmed that the natural_ground semantic segmentation channel is indeed supported – it’s just missing from the ChannelName enum as of ARDK 2.1.0 and will be re-added in a future release.

7

@rob_link Thank you for the information regarding semantic channels.

I’ve been looking into implementing the change whereby the Gameboard is created say 0.1 units above the ground plane in an attempt to reduce flickering. Could you provide some guidance if possible? Which object’s y position should I be manipulating to most easily achieve this? The whole Gameboard? The Surface object? The individual Tile positions?

8

For anyone else who might want to achieve this effect, you can just add it in GameboardModel.cs under the Scan() function. Specifically

          // Raycast for height
          var elevation =
            Physics.Raycast
            ( new Ray(position, Vector3.down),
              out RaycastHit hit,
              rayLength,
              layerMask: Settings.LayerMask
            )
              ? hit.point.y + 0.1f
              : -100;

Change " + 0.1f" to the distance above the ground you want it to be. Naturally you can edit the function to pass in this height offset. But just FYI even if this works there’re other considerations to make if you have an elevated gameboard (e.g. shadow casting) depending on your application…

@rob_link This method does help with Z-fighting that seems to affect my iPhone 12 Pro very badly. But in the video I posted at the start of this thread I think it has more to do with the calculated depth height being different from the depth texture, causing plane detection to go a bit haywire. Think I’ll just wait and hope this gets improved in future ARDK versions.

2 Likes
9

@Kang-An

Thank you for sharing your findings! I’ve passed this information along to the team.