Wimi Hologram Academy: Research On The Future Network Technology Requirements Of Holographic Communication

WIMI Hologram Academy, working in partnership with the Holographic Science Innovation Center, has written a new technical article describing their exploration of the research on the future network technology requirements of Holographic Communication. This article follows below:

AI and ML News: Why SMBs Shouldn’t Be Afraid of Artificial Intelligence (AI)

With the advent of 5G commercialization, new network technologies such as the Internet of Things, artificial intelligence, AR / VR, tactile network and holographic multimedia have gradually become a reality, and will be popularized and applied on a large scale in the next 30 years. In the stage of rapid network development, more and more emerging applications spring out. At the same time, these applications also bring higher demands and challenges to the network. For example,when we mentioning the rapid development of the Internet of Things and the proposal of “Industry 4.0”, it is necessary for the network to access the tactile network applications represented by remote surgery, the popularization of artificial intelligence such as robots and drones, set high requirements for the security and reliability of the network; AR / VR needs the network to provide a lower latency to ensure user immersive experience.Scientists from WIMI Hologram Academy of WIMI Hologram Cloud Inc. , discussed the application of holographic communication demand for future network in the 5G era.

1 Holographic communication

In recent years, with the development and application of holographic technology, holographic communication is gradually becoming possible. Holographic display technology records the phase and amplitude of the scattered light waves on the object surface by using the interferometry, and then resorts to the diffraction principle. Holographic communication is a new communication mode that uses holographic display technology to capture the images of people and surrounding objects in remote positions, transmit holographic data through the network, use laser beam projection at the terminal, project real-time dynamic stereoscopic images in the way of hologram, and can interact with it. The information media of holographic communication is the hologram. In the future, the hologram will become the information media with the most applied network applications. Holographic communication services can be widely used in remote communication, online education, healthcare, game and entertainment, commercial advertising and other fields. For example, the remote video application holographic presence remote participants as a hologram projection to the room local meeting participants, remote barrier application technicians and remote location of artifacts holographic rendering interaction, remote training and online education can provide users from remote location with surreal holographic dynamic interaction object, in order to achieve the purpose of teaching.In addition, robots used for remote surgery and immersive entertainment, games and sports also involve holographic technology.

With the continuous development of new technologies such as XR (extended reality), digital twin and artificial intelligence, the increasingly expanding business needs of holographic communication can no longer be met, and it will become more and more difficult to realize the immersive holographic service based on the current 5G communication network. Future networks need to provide greater connectivity to ensure that people can enjoy a fully immersive holographic interactive experience at any time and place, realizing the communication vision of “holographic connection”.

2 Network demand

Different from traditional media communication, holograms used as holographic communication information media, especially dynamic holograms, often contain a large number of complex data.These data are from video, audio, tactile data, and from people, physical objects, and background environment. Especially in multi-user, multi-data flow scenarios, the original network bandwidth required for holographic communication will be huge, and usually requires efficient coding / decoding technology to compress the data of the hologram.

Holographic communication usually involves multi-dimensional information, especially in the generation and transmission of dynamic hologram, prone to interference and distortion by noise, jitter and packet loss; and the system and equipment must perceive the user state and movement in real time, quickly process and respond to the received information; the information of all dimensions must be strictly synchronized, so that the user can feel the reality and immersion of interaction.Therefore, the delay requirement of holographic communication is much more stringent on the network than on the traditional AR / VR interactive applications.

In addition, holographic communication also has Quality of Service (QoS) requirements. It is able to identify key network parameters and select different transmission protocols based on data priority and QoS, and can even express QoS requirements to the network. Since holographic communication services involve a large number of user privacy data, how to protect users’ privacy data in the communication process is also an urgent problem to be solved.

2.1 Ultra-high bandwidth

Compared with traditional HD and 3D virtual video, the holographic communication transmission will reach Mbit/s.A 1080P image with 4byte color data per pixel, a depth image resolution of 512dpi 424dpi and 2byte depth data for each pixel, equivalent to the raw data of 70.4MB per frame.And, as the number of sensors and viewpoints increases, the network bandwidth is higher at higher resolution and frame rates.

Based on the size of the projected object and the frame rate, the holographic communication requires a network bandwidth of 1 Gbit/s ~ 1 Tbit/s.Among the three application scenarios defined by 5G, eMBB provides a user experience rate of 0.1 ~ 1 Gbit/s and a peak rate of dozens of Gbit/s. For holographic communication in HD and above, 5G network is a little “powerless”. Using more efficient image compression technology and codec scheme can alleviate the bandwidth demand of holographic communication to a certain extent, but the future network still needs ultra-high bandwidth.Studies on higher working frequency bands such as millimeter wave, terahertz and visible light show that future networks can provide a user experience rate of 100 Gbit/s and a peak speed of more than 1 Tbit/s.At the same time, developing these new spectra poses greater challenges for antenna and RF technologies.

2.2 Ultra-low time delay

With the same requirements for strongly interactive immersive applications such as AR / VR, holographic communication requires the web-network to provide end-to-end delays of less than 1ms. The end-to-end delay provided by the 5GuRLLC（ultra-reliable and low latency communications）application scenario is ms level, theoretically 5G transmission delay up to 1ms, but it is almost impossible to achieve in long-distance communication due to the physical limitations of the speed of light.The transmission protocol used by traditional networks fails in the fold between delay and deterministic to satisfy applications that require immersive interactions.

The emergence of RDMA technology solves the delay caused by client and server-side data processing in network transmission. It transfers data directly from one computer’s memory to another, without the involvement of both operating systems. Compared with the traditional TCP / IP communication mode, RDMA allows for high throughput, and low latency of network communication and this technology can further reduce the network transmission delay, which has great development potential in future network application scenarios.

2. 3 Network computing power

In order to truly realize the process of holographic communication, first, obtain the object information through the acquisition end device, calculate and generate the hologram, conduct network transmission after coding compression, and reconstruct the hologram of the object after the terminal decoding and display it. Because the hologram contains a huge amount of information and data, and the calculation time is too long, it will not only bring a great bandwidth burden, but also cause a great MTP delay. To meet the user immersive experience, for strong interactive applications such as AR / VR, MTP delay is less than 20ms, and holographic communication is 10ms or even lower.

In terms of hologram compression, because holograms contain a large number of complex data, efficient coding technology is needed to compress holographic data. Traditional video coding technology is not fully applicable to holographic data. At present, the main coding technology used for HD video is H.264, and HD video above 8K needs to be upgraded to H.265. On the premise of ensuring the same picture quality, the HEVC / H.265 compression ratio can be increased by about 30% compared with H.264, up to 600:1. Recent research progress in standard organizations such as MPEG shows that the compression ratio of the corresponding next-generation coding technology VVC / H.266 can increase by about 40%.

In terms of hologram generation, the future network needs to provide a large capacity channel for real-time acquisition and data transmission at the acquisition terminal; eliminate the computing modules and data storage modules in the computing terminal to shift the computing power requirements of the devices to the cloud; The transmission side requires high-speed and stable network connection through wireless mode; maintain stable terminal and cloud connection at the display end to make the delay reach ms level and ensure immersive experience.

With the rapid development of cloud computing and MEC (mobile edge computing) technology, the future network can solve the computing power needs of holographic communication through the rapid deployment in the cloud and at the edge.

2.4. Synchronicity and QoS

The generation and transmission of holograms contain multiple dimensions of information, derived from video, audio, touch, smell, taste, etc. Only when the information of all dimensions is strictly synchronized can it give the user a sense of immersive experience. Therefore, quite strict synchronization needs to be maintained between the various concurrent media streams that generated holograms from objects of different sensors and different angles during transmission.

For holographic communication services, multiple channels are involved, and each channel can map a single flow with strict time requirements and different elasticity requirements. The aggregate resources of holographic data can be shared and redistributed among channels, and the waiting time across channels is not to exceed 7ms. Existing networks lack basic service capabilities with sufficiently low latency and sufficient high bandwidth, and existing technologies do not support the concept of aggregated bandwidth shared and dynamically redistribution among a set of streams.

Furthermore, the network needs to support and have the ability to dynamically adapt to multiple concurrent streams. According to the size of the cloud and image array, the network needs to support at least 1,000 concurrent streams. How to intelligently manage the QoS of these concurrent streams is a considerable problem and test for the future network.

2. 5 Reliability

Gathering and packet loss can to data distortion of the hologram, causing crosstalk, thus affecting the user’s experience. For immersive applications, the jitter caused by the unstable network environment will greatly reduce the immersion sense of these applications, so every layer during the network transmission process should ensure the deterministic latency and a very low packet error rate. The maximum packet error rate provided by a 5G network is on the order of 105, AR / VR strong interactive application requires 106, and holographic communication pair requires a packet error rate of at least 107.

2.6 Safety

The hologram transmitted through holographic communication contains a large amount of information data, including facial features, voice, and other sensitive information, which requires the network to provide an absolute security guarantee, and the use of existing security technologies will increase the end-to-end delay. To provide secure transmission with extremely low latency, one scheme is to embed communication security for eavesdroppers and attackers into the physical transmission, ensure that only legitimate receivers can process control and feedback messages, and develop new and secure codec schemes for novel tactile messages. The compromise consideration of delay and security is one of the difficult problems for networks in the future.

3 Opportunities and challenges

The rapid development of the network has brought great convenience to science and technology and life. In many interactive applications, holographic communication provides holographic video call, immersive shopping, remote holographic surgery, and other services, to satisfy people to enjoy fully immersive holographic interactive experiences and realize the deep integration of virtual and real scenes. However, the ultra-high bandwidth, ultra-low latency, network computing power, synchronization, QoS, reliability, and security indicators required by holographic communication cannot be realized by the current network architecture and technology. Future networks need breakthroughs in system architecture and network technology to support these emerging and demanding future applications.

Top Artificial Intelligence Insights: Determining the Potential of Your AI Algorithm Starts with Measurement

In terms of network architecture, to solve the increasingly exposed problems of the current TCP / IP architecture, countries around the world have implemented several studies on the future network-related and proposed two “evolutionary” and “revolutionary” technology routes. Among them, the “evolution” technology route advocated in the existing TCP / IP network architecture patch repair, put forward new network technology, change the existing network form, network equipment or topology, to upgrade the existing network communication protocol, and artificial intelligence, blockchain, big data and other new technology applied to the network, so that the existing network architecture to adapt to the new development needs. “Revolutionary” technology route is advocated all over again, namely, based on the existing network constraints into new network architecture, redesign network communication protocol, and define it as the future network architecture, to overcome the traditional network architecture in scalability, security, controllable and QoS security, mobility, service distribution, and green energy-saving, better adapt to the needs of future development, realize the sustainable development of the network.

Industry controversy about the two technology routes, “evolving” technology routes will make the originally simple network structure increasingly complex and bloated, while “revolutionary” technology routes will slow down the pace of actual deployment. With the research of NFV, SDN, SR, ADN, SCN, MEC, artificial intelligence, IPv6, blockchain, and other new network technologies, what architecture and standards should be adopted for the future network is still the focus of intense debate in the industry.

Today, the development of network application has gradually become the demand source of network technology development, the research of new network technology and system architecture is to better support the future network application, and clear future network application performance requirements in turn will provide a clearer direction to network architecture research, these will be for science and technology, system architecture, emerging service application development provides opportunities and challenges.

Founded in August 2020, WIMI Hologram Academy is dedicated to holographic AI vision exploration and researches basic science and innovative technologies, driven by human vision. The Holographic Science Innovation Center, in partnership with WIMI Hologram Academy, is committed to exploring the unknown technology of holographic AI vision, attracting, gathering, and integrating relevant global resources and superior forces, promoting comprehensive innovation with scientific and technological innovation as the core, and carrying out basic science and innovative technology research.

Read More About AI News : AI Innovation Supports Rural and Remote Internet Connectivity

[To share your insights with us, please write to sghosh@martechseries.com]