The author of this article is Sun Xuejing, CEO of Time Turing, who has finished his lecture at WorkSEhop at ICEVE (Beijing International Advanced Imaging Conference & Exhibition).
Why is panorama sound the standard of VR? I don't know how much everyone knows about panoramic sound and VR. For a long time, everyone pays attention to VR. It may be video, 4K, or the quality of camera. I don't mention it, I believe that everyone has done dubbing work before the film school. The voice is really important in film and television, but why do we emphasize it today, especially in VR? Let's talk about it next. We are aware of it.
What is panoramic soundThe term panoramic sound may not be very strange to everyone, because it has appeared in recent years, including Dolby's panoramic sound, so I will talk about the difference between it and traditional audio. First of all, panoramic sound is a full-scale display. Different from stereo and surround sound on a horizontal plane, panoramic sound definitely needs a full 360&TImes; 360 display form, and then it is very important that we think it needs Supporting interaction, in a sense, very closely related to games, or panoramic movies, is that it is not static, citing many concepts of interaction.
The difference between panoramic sound and 3D soundMaybe everyone will ask what is the difference between panoramic sound and 3D sound. We think that panoramic sound is much larger than 3D sound. 3D sound can be considered as a subset of panoramic sound, but it is true that there are many similarities in core technology. local.
From the development of the sound we briefly recall that the tradition is mono, especially in the communications industry, and many mono channels have been used for almost a few decades. Most of the stereo is used to listen to music, this is also a certain reason, because the balance of people when we listen to the song is still very important, so in film and television, commercial is 5.1/7.1, mainly Dolby First of all, it has achieved great success in business. In fact, there are some different disputes in this system when it appears. Finally, 5.1/7.1 is a commercial one that can meet the needs of the public. Traditionally, the technology that we said today is the predecessor of Ambisonics, which is more popular, so it should have dominated the business for more than 10 years.
Now, after 5.1/7.1, everyone even heard 13.1 and 22.1. These are some theaters, including NHK Japan. They have this technology of more speaker tracks. After the latest channel-based technology. What technology are you paying attention to in the audio field?
At present, internationally, in terms of standards, it is led by Dolby. It is pushing Dolby Atmos, and then another camp is represented by Fraunhofer and Qualcomm. They represent the MPEG H camp and propose new standards. Instead of the codec of this traditional multi-track AAC, when they pushed these standards a few years ago, VR was not particularly hot at the time, but its core technology has begun to introduce the concept of Object Audio, and then in MPEG H. In the meantime, it introduces the high-end Ambisonic concept, but at that time it was more oriented to the cinema industry and was not directly linked to VR, so it did not consider much in terms of interaction.
The sound in the VR video needs to be orientedThe interactive feature I just mentioned means that when we look at VR content, we are actually pursuing a very good sense of presence and presence. The so-called experience of better authenticity is essential for interaction with people. This immersion is not enough if you just passively experience the content that the director presents to you.
Everyone has experienced the VR helmet, and you will at least turn around and see different panoramic content, and the sound will change. In a more advanced way, it is similar to the inside of the game. You enter from one room to another or you are in a room. You are close to the corner of the room, or away from this corner. The sound you hear is actually not In the same way, this is exactly the same as the visual change, because the introduction of these interactive features has led to the traditional and 5.1 mixes that cannot meet the interactive needs.
We also need VR panoramic sound technology. As an engineering system, it needs to support headphone playback, because it is mainly based on headphones in VR, but the speaker also needs support, because it will have another application scenario. These support across platforms, supporting different VR devices is also a must for us as a commercial company to do a commercial system.
Some VR related recording technologies
In general, the entire panoramic sound system is actually a relatively important part of the collection, production, and playback rendering. If you understand the video, it is the same analogy. You also need to compare the video from the capture and editing to the final playback.
Let's start with some recording techniques. There are several mainstream technologies for recording. The traditional one is stereo recording, such as XY recording, or AB recording. It is usually two microphones, which can be located close to each other, and it will collect spatial sound on a plane. The binaural recording is actually a special place in stereo. It uses an artificial head to simulate the characteristics of a person's head, and then inserts some omnidirectional microphones on the left and right sides of the human head, so it will The 3D sound field has a very realistic simulation. Playing this sound is basically played through the headphones, no additional codec or format conversion is required.
HRTFHRTF is one of the most critical and core technologies in our 3D sound effects or in sound simulation. The underlying math is not particularly complicated. It uses some wired FIR filters to simulate the transition from head to source. The signal path of the information transmission is then simulated with some simple numbers.
At present, the method we use is that I use a human head to collect a lot of human head transfer functions from various directions in a muffler room. For example, I put a sound source in front of the human head at one meter, and I sweep the zero angle. Frequency, or white noise and pink noise, I can repeat it many times. I will collect it at different angles and different heights, then record the sounds and store them in digital form after some mathematical analysis. If I need to simulate these 3D sound effects in the future, then I will bring these math functions into it, and of course I am doing a particularly good premise. This actually requires a lot of mathematics, a lot of engineering considerations, such as the size of our head, etc., in fact, this is a very serious problem.
Now we are directly related to VR, we call it Omni-Binaural. As you can see from the picture, it is an upgraded version of the traditional head recording, which is two artificial ears with positive and negative directions in all four directions. In fact, this equipment is produced by foreign 3Dio company, which saves the human face to simulate the human head, which is actually a simplified version. Of course, I have also seen that a foreign team is doing a model with a little horror. In fact, every face has a face. I can imagine that there is a bit of horror. But it will work better than just using the human ear.
If we use this device to record, we can actually see two microphones on each side, so the last thing you get is 8-track sound, so it simulates four directions, four degrees, 0 degrees, 90 degrees, 180 degrees, 270. Degree, if you want higher precision, you can actually add more faces. Now basically the simpler way is to simulate with four sides, that is, people can hear a particularly good person in different directions. recording.
Its advantage is that decoding is relatively simple, then what if I want to listen to 45 degrees? The way it actually does is to make a mathematical difference between the sound recorded at 0 degrees and the sound recorded at 90 degrees, so it is actually very simple. But the effect that comes with it will be discounted. It will be slightly blurred when you hear the sound at 45 degrees, which is also its flaw.
AmbisonicAmbisonic technology has become an important technology since the rise of VR, so it is traditionally based on this relatively expensive microphone, such as the Soundfield of TSL, which basically places the microphone. The location is a so-called tetrahedron, Tetrahedron's microphone is placed, you can imagine that it is actually each microphone in one direction, there are upwards, left, right, and horizontal directions, it can put 360 The sound heard at one point is included.
So these products TSL, TetraMic has been around for a long time, and then Senhai's Ambeo is just coming out this year, these models are very high quality, just an array of microphones, if you need to use this device to collect, It is also necessary to use this recording device extra. We usually call it an orbital machine, which can support multiple channels of microphones.
We can see that usually the first-order Ambisonic is represented by WXYZ. If it is recorded with Zoom, its Z-axis is zero, only WXY. The second device is the Twirling720 product launched by our company's era Turing. This product is a four channel, four tracks are all component, so it is a more complete first-order Ambisonic recording device.
I just briefly introduced two devices that we mainly use for panoramic sound recording, especially in VR. One is Omni-Binaural, the other is recording equipment based on Ambisonic technology, and of course there are some traditional recordings, such as 5.1 surround sound, and Just mentioned XY, or AB such stereo recording.
When you have finished recording, it involves the transmission and storage of panoramic sound. At present, I personally think that this is the biggest problem that the industry has not solved. Once involved in transmission and storage, it requires a strong format uniformity. But on the other hand, as long as the format is mentioned, it involves a lot of commercial interests, and it is very difficult to unite each other.
The Ambisonic technology is also likely to be used more and more after the emergence of VR. This is also a format compatible with Google or Facebook. Because this format has no patent restrictions, it is relatively open. If you follow an Ambisonic format then I can guarantee that my video will be available on YouTube or Facebook. But this format is actually a bit like Object Audio or Channel-based Audio. I personally call it a presentation, not a final codec, because no matter which technology, the underlying codec. Can be encoded with MP3 or AAC, this specific coding form is actually another lower level of technology.
How to play panoramic sound?Here is the use of Ambisonic in the VR field. The core usage is called the intermediate format. That is, regardless of the acquisition format or the format made by the track, it can be packaged into the Ambisonic format, which is just said. WXYZ, and finally turn it into 5.1 or stereo, or support rotation. So everyone will have a mechanism to shake hands, there is an interface, so it is easier to handle, if it comes in Dolby panoramic sound format, in fact, many brands can not play.
But YouTube and Facebook can broadcast such content that supports SPATIal 360 and SPATIal audio. Like YouTube, it is based on Ambisonic. Then Google recently launched a web-based player called Google Omnitone, which is also based on Ambisonic decoding, so if you The content is the Ambisonic sounding device just mentioned, like Twirling720, then you can upload to YouTube or Google Omnitone supported webpages with one click, so now these websites can already see a lot of 360 audio content, Facebook is also supported. .
In addition, like the format of Quad Binaural, if you use this multi-channel head recording Omni-Binaural device, the platform it supports should be GearVR, and there are other platforms. Of course, the time of the Turing game is also supported. However, its shortcoming is that the number of audio tracks is relatively large, and it is not so good in platform compatibility. As for Object Audio and Dolby Atmos, it will be more limited, especially if you don't do it. Some conversions, then its complexity is also high.
In terms of the playback platform, the domestic market is still slower than foreign countries. Of course, our own platform can support Ambisonic panoramic sound playback, such as App (Turro VR), official website () and YouTube personal homepage (twirlingVR). We also hope to change this form by collaborating with Youku, Orange VR or other VR vertical field players to play VR content that supports 360 degree SPATIal audio on their platform as soon as possible.
The importance of panoramic sound
In the VR experience, we first talk about panoramic video. Now that the video everyone thinks it is almost the same, what should be done on the audio next? This is actually a problem that many people are paying attention to in the second half of the year. At present, most of the domestic filming is based on traditional audio, and a few teams have begun to study the concept of this panoramic sound.
Traditional audio is actually a good sound rendered in advance, whether you are listening to music in stereo, or watching a movie in a theater, then listening to 5.1, these are the sounds mixed once and never change, so it does not have much interaction. the concept of. In VR, we actually need strong interaction, so it puts more stringent requirements on traditional audio algorithms. Traditional 5.1 is more complicated but mixed once is fixed, playback is very fixed, so there is not much efficiency. problem.
However, in the panoramic sound, if it is to completely reproduce the scene in real life, such as having a particularly complicated scene, the amount of calculation may be very high, you need any action, turn around, bow, move, both To recalculate that the sound in this sound field reaches the human ear, you should need the correct transfer function, causing all the operations to be recalculated. The amount of this operation is actually very large.
I also honestly say that the ultimate goal of achieving this is definitely not achieved on mobile phones. What we are doing now is how to ensure good quality in a particularly efficient situation. This is what we have established since last year. The company has been investing a lot of energy in research and development until now.
Panorama sound is the need to interact to give you a better sense of presence, and then another important point is that through the sound I can give the director more tools to show his intentions, the user can better understand the director through the voice expression.
In VR video, under the guidance of no sound, there may be several concerns in the picture.I believe that everyone has seen the panoramic video. If there is no sound prompt, then you are watching the video according to your own preferences when you watch it. I guess what this panoramic video is expressing. I think this is very inefficient. All of these very good VR works taken abroad have, without exception, their sounds are very good. They all have the idea of ​​guiding the audience to see the content through the advantages of panoramic sound.
Over time, battery cables have to be replaced due to corrosion or damage. A bad battery cable can cause intermittent starting issues or lack of power to the vehicle, including arcing or power drains.A battery cable consists of multiple stands of wire encased in synthetic material with different types of battery terminals on each end for a reliable connection. Corrosion is the number one cause of battery cable failure, stopping the flow of electricity.
Battery cable, power cable, battery cable assembly, battery wiring
ETOP WIREHARNESS LIMITED , https://www.oemmoldedcables.com