If you’re doing this to achieve a visual effect, you could try using CreateOrUpdateTextureARGB32() and adding a custom shader as done in the depth textures tutorial, but then modifying the frag color in some way.
I tried modifying the Fragment color which then changed the Background color from black to Red.
As far as I understand, we need to apply some interpolation to the depth values in the range 0 to 1. For instance, Turbo colormap as in ARCore. But their implementation seems more involved as I don’t have any experience in Shaders. So far, I was looking for simpler ways to modify the fragment shader with no luck.
While we look into your request, would you be able to explain your use case for needing to convert the depth map to color? Since the coloring in a depthmap in only in black and white since it just represents the depth values of pixels in an image, converting them to color might not yield very good results.
Thanks for your response.
We wanted to compare it with a different depthmap coming from ARCore.
Since we don’t know exactly which Depth estimation model they are using, its hard to evaluate with depth evaluation metrics.
So one of the option was to generate depth with Turbo colormap similar to ARCore and check visually.
We’re looking into this, but I’m currently not aware of a function in ARDK that does what you are describing. I’ve reached out to engineering to confirm in the case that I’m missing something.
In the meantime, would you be able to provide any links, or other information on the procedure that you use with Turbo colormap and ARCore?
Applying a custom colormap to the depth isn’t something we have direct support for, but I can give you some guidance as to how to do the conversion. A colormap is effectively a lookup table which converts a grayscale value to a color image. For every grayscale value there is a corresponding rgb value. You can find links to an example lookup table and polynomial approximation in this blog post about Turbo: Google AI Blog: Turbo, An Improved Rainbow Colormap for Visualization. To do this mapping and render it I will point you to the link @Dan_Yu posted above. This link explains how to get the depth texture and use it in a shader. In this case you would need to write a shader which converts the grayscale values to color values using the lookup table or the polynomial approximation.
There’s a caveat to be aware of which will affect your comparison. There are a number of different ways of mapping grayscale values to the colors even with the same colormap and I am not certain which way ARCore does this. Without knowing this, it won’t be possible to generate the colormap from the grayscale values in the same way and thus the comparison won’t be accurate. If you do not have access to the metric depth from ARCore or the exact method they use to generate the colormap then it won’t be possible to do a fair comparison. If you can explain to me more about why you wish to do this comparison, I can help you further.
Thanks for getting back to my query. I had tried to decode how Google applies Turbo colormap before posting this question. However, it seems a bit involved and with minimal experience in shaders it seemed like a time consuming task. So I found a different way to compare the two depth maps by digging into the details of depth evaluation metrics. So its not a required feature at the moment.