Video surveillance compression technology and video technology past and present

At present, various types of compression type are MPEG1--MPEG4--MPEG7-MPEG21-H.264 MPEG is an abbreviation of MovingPicturesExpertsGroup (Moving Picture Experts Group). Founded in 1988, this group of experts is responsible for establishing video and audio standards for CDs, and its members are technical experts in the fields of video, audio and systems.

The earliest MPEG creators originally planned to develop four versions: MPEG-1, MPEG-2, MPEG-3, MPEG-4, to suit the requirements of different bandwidth and digital image quality.
Later, due to the excellent performance of MPEG-2, it can be applied to HDTV, so that MPEG-3 originally designed for HDTV has been abandoned before it was born. So there are only three versions available: MPEG-1, MPEG-2, MPEG-4. If MPEG-1 "small file, but poor quality"; and MPEG-2 "good quality, but more space", then MPEG-4 is a good combination of the advantages of the first two. It was finalized in October 1998 and became an international standard in January 1999. It was followed by a second edition for extended use and ended at the end of 1999.
MPEG-2 technology features: MPEG-2 was developed in 1994 and is designed to meet the image quality of advanced industry standards and higher transmission rates. MPEG-2 can provide a transmission rate between 3MB and 10MB/s, and a resolution of 720×486 in NTSC. MPEG-2 can provide broadcast-quality video and CD-quality sound. MPEG-2 audio encoding provides both left and right and two surround channels, as well as one booster bass channel and up to seven audio channels. Another feature of MPEG-2 is that it provides a wide range of variable compression ratios to accommodate different picture quality, storage capacity and bandwidth requirements. Applications: MPEG-2 technology is the standard technology for implementing DVDs, and now DVD players are beginning to become popular in the home. In addition to being the designated standard for DVDs, MPEG-2 can also be used to provide broadcast-grade digital video for broadcast, cable television networks, cable networks, and satellite live broadcasts. 2. MPEG-3 has been able to be applied to HDTV (High Definition Television) due to its excellent performance, so that MPEG-3 originally designed for HDTV has been abandoned without being born.
MPEG-4 technology features: MPEG-4 was released in November 1998. It not only targets video and audio encoding at a certain bit rate, but also pays more attention to the interactivity and flexibility of multimedia systems. This standard is mainly applied to video telephony, video e-mail, etc., and has a low transmission rate requirement of 4800-64000 bits/s and a resolution of 176×144. MPEG-4 utilizes a very narrow bandwidth, through frame reconstruction techniques, data compression, to achieve the best image quality with the least amount of data. Scope of application: After two years of development, the most popular application now is to convert MPEG-2 video files in DVD into smaller video files by using MPEG-4's high compression ratio and high image restoration quality. After this processing, the video quality of the image is not greatly reduced, but the size can be reduced by several times, and the CD-ROM can be conveniently used to save the program on the DVD. In addition, MPEG-4 will be of great use in home photography video and web real-time video playback. MPEG4 is mainly used in videophones, videomails, and electronic news, and has a lower transmission rate requirement between 4800-64000 bits/sec. With a very narrow bandwidth, the data is compressed and transmitted through frame reconstruction techniques to achieve the best image quality with minimal data. The feature of MPEG-4 is that it is more suitable for interactive AV services as well as remote monitoring. This is an interactive dynamic image standard. From the current situation, MPEG-4 is likely to be used in three areas: digital TV, interactive graphics applications (including content synthesis), interactive multimedia, and so on. MPEG-4 provides standard technology that enables it to be integrated into products, categories, content access and more. Following MPEG-4, a series of technologies have been provided to address the growing image and sound MPEG-4 standards to meet the needs of authors, service providers, and end users. For the author, MPEG-4 can guarantee a great deal of content reusability. It is more flexible than the technologies we see today, such as digital TV, animation, etc. For network service providers, MPEG-4 provides technology. To support the ability to interpret or translate information into appropriate localized information; end users, MPEG-4 can bring more interactivity.

As we mentioned earlier, there are two versions of MPEG-4. The second version is based on the first version and is backward compatible. In general MPEG-4 provides a standard way to describe the scene, where the description of the scene relies on virtual, which is the concept of the MPEG-4 standard in the modeling language (VRML) consists of the following parts.

1. DMIF (TheDellivery Multimedia Integration framwork) is the overall framework of multimedia transmission. It mainly solves the operation problems of multimedia applications in interactive networks, broadcast environments and disk applications.

2, the data plane, the data plane in MPEG4 is divided into two parts: the transmission relationship part and the media relationship part. In order for the elementary stream and the AV object to appear in the same scene, MPEG4 refers to the concepts of object description (OD) and stream map desktop (SMT).

3. Buffer management and real-time identification, MPEG4 defines a system decoding mode (SDM). This decoding mode allows for better utilization of limited buffer space by efficient management.

4, audio coding, MPEG4 not only supports natural sound, but also supports synthetic sound.

5. Video coding, similar to audio coding, MPEG-4 also supports the encoding of natural and synthetic visual objects.

MPEG-7 MPEG proposes the solution MPEG-7. The work was proposed in 1998 and was finalized in early 2001. MPEG-7 will standardize the description of various types of multimedia information for fast and efficient search. The standard does not include automatic extraction of descriptive features, nor does it dictate tools or any programs that use the description to search. Its official title is "multimedia content description interface." MPEG-7 can be used independently of other MPEG standards, but the description of audio and video objects defined in MPEG-4 applies to MPEG-7, and this description is the basis of classification. In addition, we can use the description of MPEG-7 to enhance the functions of other MPEG standards. Overall, MPEG has three advantages. First of all, it is developed as an international standard, so it has good compatibility. Second, MPEG can provide a better compression ratio than other algorithms, up to 200:1. More importantly, MPEG provides a high compression ratio while minimizing data loss. Compared with the AC series standard, which is also an audio compression standard, the MPEG standard series is more suitable for promotion because there is no patent problem. MPEG-1 makes VCDs replace traditional video tapes; MPEG-2 will eventually make digital TVs completely replace existing analog TVs; with the new standards of MPEG-4 and MPEG-7, data compression and transmission technologies will surely The trend is more standardized. To be precise, MPEG-7 is not a compression coding method, but a multimedia content description interface. After MPEG-4, the contradiction to be solved is the management and rapid search of increasingly large images and sound information. MPEG7 is the solution to this contradiction. MPEG-7 strives to quickly and efficiently search for different types of multimedia image data that users need, such as searching for images of the Yangtze River Three Gorges lens in the image data. This program was finally completed and announced in early 2001.
MPEG-21 MPEG-21 will be developed from MPEG-7 and has only just begun. According to reports, MPEG-21 mainly regulates online real-time exchange protocols for digital programs.
Related video formats After reading the audio and video compression format of the MPEG series, I think everyone must also care about the development of other related compression formats, as well as the advantages and disadvantages of each place. I think that only by knowing this can we better grasp the overall situation. ASFASF is the abbreviation of AdvancedStreamingformat, which is a file compression format developed by Microsoft to compete with RealPlayer to watch video programs directly on the Internet! Using MPEG-4 compression algorithm, as a video "stream" format for instant viewing on the Internet, it is a little bit worse than VCD, but better than other video "streaming" format RAM format. If you don't consider spreading it online, choose the best quality to compress the file, and the resulting video file is better than the VCD. But in this case, it lost the original intention of ASF. nAVInAVI is an acronym for newAVI, but this is not a development from Microsoft. It is a new video format developed by an underground organization called ShadowRealm. It is modified by the Microsoft ASF compression algorithm, sacrificing the video stream characteristics of ASF, and improving some of the shortcomings of the original ASF format. Simply put, it is a non-network. Version of ASF! The international standards organization MPEG is about to approve a new digital video compression format that is expected to bring improvements to emerging multimedia technologies, but it also adds some uncertainty. The new format is called H.264. It is said to be able to send DVD-quality video over the Internet, using significantly less network resources than its competitors.
According to Robert Koenen, the format (or codec for multimedia digital signals) was created by the "Joint Video Team" formed by the United States and the European Standards Organization and will be part of the MPEG-4 (Moving Picture Experts Group) multimedia standard at the end of the year. Obtained. MPEGLA, the main license exchange center for the MPEG-4 standard, has asked companies to submit patents that they consider to be in the H.264 format by Friday for their consideration. Compressing large amounts of data files is the key to enabling online video delivery and delivery to wireless devices. Media companies have long been tempted by these two markets, but some have failed to do so because of cost and quality issues. Few high-speed Internet access providers can guarantee data transfer speeds of more than 500 kbit/s, making video file size the biggest barrier to Hollywood's Internet video distribution program. H.264 took a long time to solve this problem. At the time of the test, the playback quality reached the DVD level at a transmission speed slightly lower than 1 megabit per second. While this does not mean that the average consumer will soon be able to enjoy a DVD-quality video stream over a standard broadband connection, it sets an important performance benchmark compared to other formats. The data savings achieved by H.264, MPEG-4 Part 10, can speed up the development of Internet and wireless video on demand services. It may prove to be very valuable to cable operators who want to play more channels through their channels and to issuers who are looking to load more high-quality video files into digital media such as DVDs.
Currently, these industries are still using the older MPEG-2 video standard, which is four times larger than the new standard. H.264 is also expected to increase performance by 33% over current video formats implemented with MPEG-4. H.264 is a new digital video coding standard developed by ITU-T's VCEG (Video Coding Experts Group) and ISO/IEC MPEG (Moving Picture Coding Experts Group) Joint Video Team (JVT: jointvideoteam), which is both ITU -T's H.264 is the 10th part of ISO/IEC MPEG-4. The draft was drafted in January 1998, the first draft was completed in September 1999, its test mode TML-8 was established in May 2001, and the FCD board of H.264 was adopted at the 5th meeting of JVT in June 2002. . Officially released in March 2003. H.264, like the previous standard, is also a hybrid coding mode for DPCM plus transform coding. However, it adopts the simple design of “return to basics”, and does not use many options to obtain much better compression performance than H.263++; it has enhanced the adaptability to various channels, adopting “network-friendly” structure and syntax. It is beneficial to the processing of error and packet loss; the application target range is wide to meet the needs of different speeds, different resolutions and different transmission (storage) occasions; its basic system is open and use without copyright. Technically, there are multiple flashes in the H.264 standard, such as unified VLC symbol encoding, high-precision, multi-mode displacement estimation, integer transform based on 4×4 blocks, hierarchical coding syntax, and so on. These measures make the H.264 algorithm have a very high coding efficiency, and can save about 50% of the code rate than H.263 under the same reconstructed image quality. H.264's code stream structure network has strong adaptability, increases error recovery capability, and can adapt well to IP and wireless network applications. H. Technical Highlights of 264 1. Hierarchical Design The H.264 algorithm can be conceptually divided into two layers: the video coding layer (VCL: VideoCodingLayer) is responsible for efficient video content representation, and the network abstraction layer (NAL: NetworkAbstractionLayer) is responsible for the network. The data is packaged and delivered in the appropriate way. A packet-based interface is defined between the VCL and the NAL, and the packing and corresponding signaling are part of the NAL. Thus, the tasks of high coding efficiency and network friendliness are performed by VCL and NAL, respectively. The VCL layer includes block-based motion compensated hybrid coding and some new features. Like the previous video coding standards, H.264 does not include pre- and post-processing features in the draft, which increases the flexibility of the standard. The NAL is responsible for encapsulating data using a segmentation format of the underlying network, including framing, signaling of logical channels, utilization of timing information, or sequence end signals. For example, NAL supports the transmission format of video over circuit switched channels, supporting formats in which video is transmitted over the Internet using RTP/UDP/IP. The NAL includes its own header information, segment structure information, and actual payload information, that is, upper layer VCL data. (If data segmentation is used, the data may consist of several parts). 2. High-precision, multi-mode motion estimation H.264 supports motion vectors with 1/4 or 1/8 pixel precision. A 6-tap filter can be used to reduce high-frequency noise at 1/4 pixel accuracy, and a more complex 8-tap filter can be used for motion vectors with 1/8 pixel accuracy. When performing motion estimation, the encoder can also select an "enhanced" interpolation filter to improve the prediction effect. In the motion prediction of H.264, one macroblock (MB) can be divided into different sub-blocks according to FIG. 2 to form block sizes of seven different modes. This multi-mode flexible and meticulous division is more in line with the shape of the actual moving object in the image, greatly improving the accuracy of motion estimation. In this manner, 1, 2, 4, 8, or 16 motion vectors can be included in each macroblock. In H.264, the encoder is allowed to use more than one frame of previous frames for motion estimation, which is the so-called multi-frame reference technique. For example, a 2 or 3 frame just encoded reference frame, the encoder will choose to give a better predicted frame for each target macroblock and indicate for each macroblock which frame is used for prediction. 3, 4 × 4 block integer transform H.264 is similar to the previous standard, block-based transform coding for residuals, but the transform is an integer operation rather than a real operation, the process is basically similar to DCT. The advantage of this method is that it allows for the same transform and inverse transform in the encoder and in the decoder, making it easy to use simple fixed-point arithmetic. That is to say, there is no "inverse transformation error" here. The unit of transformation is 4 × 4 blocks, instead of the 8 × 8 blocks that were commonly used in the past. Due to the size reduction of the transform block, the division of the moving object is more accurate, so that not only the calculation amount of the transformation is relatively small, but also the convergence error at the edge of the moving object is greatly reduced. In order to make the conversion method of the small-sized block do not generate the gray-scale difference between the blocks in the smooth area of ​​the large area in the image, the DC coefficients of the 16 4×4 blocks of the intra-macro block luminance data may be generated (each small block) One, a total of 16) performs a second 4×4 block transformation, and performs a 2×2 block transformation on the DC coefficients of four 4×4 blocks of chrominance data (one for each small block, four in total). In order to improve the ability of rate control, the amplitude of the change of the quantization step is controlled at about 12.5% ​​instead of the constant increase. The normalization of the magnitude of the transform coefficients is processed in the inverse quantization process to reduce the computational complexity. To emphasize the color fidelity, a smaller quantization step size is used for the chrominance coefficient. 4. There are two methods for entropy coding in unified VLCH.264. One is to use uniform VLC (UVLC: Universal VLC) for all symbols to be coded, and the other is to use content-adaptive binary arithmetic coding (CABAC: Context-AdaptiveBinaryArithmeticCoding). CABAC is optional and its coding performance is slightly better than UVLC, but the computational complexity is also high. UVLC uses an infinite number of codewords, and the design structure is very regular. Different objects can be encoded with the same code table. This method is easy to generate a codeword, and the decoder can easily identify the prefix of the codeword. UVLC can quickly obtain resynchronization when a bit error occurs. Figure 3 shows the syntax of the codeword. Here, x0, x1, x2, ... are INFO bits and are 0 or 1. Figure 4 shows the first nine codewords. For example, the 4th number word contains INFO01, which is designed to be fast resynchronized to prevent bit errors. 5. Intra Prediction In the previous H.26x series and MPEG-x series standards, the interframe prediction method was adopted. In H.264, intra prediction is available when encoding Intra images. For each 4x4 block (except for special handling of edge blocks), each pixel can be predicted with a different weighted sum of 17 closest previously encoded pixels (some weights can be 0), ie this pixel 17 pixels in the upper left corner of the block. Obviously, such intra prediction is not a temporal, but predictive coding algorithm performed in the spatial domain, which can remove the spatial redundancy between adjacent blocks and achieve more efficient compression. As shown in FIG. 4, a, b, ..., p in a 4x4 block are 16 pixels to be predicted, and A, B, ..., P are encoded pixels. The value such as the m point can be predicted by (J+2K+L+2)/4, or by (A+B+C+D+I+J+K+L)/8, and so on. There are 9 different modes of brightness depending on the point of the selected prediction reference, but the intra prediction of chrominance has only the 1 mode. (6) For the IP and wireless environment, the H.264 draft includes tools for error elimination, which facilitates the transmission of compressed video in error-prone, packet-dropping environments, such as the robustness of transmissions in mobile channels or IP channels. To combat transmission errors, time synchronization in H.264 video streams can be accomplished by using intra-frame image refresh, which is supported by slice structured coding. At the same time, in order to facilitate resynchronization after error, a certain resynchronization point is also provided in the video data of one image. In addition, the intra macroblock refresh and the multi-reference macroblock allow the encoder to consider not only the coding efficiency but also the characteristics of the transmission channel when determining the macroblock mode. In addition to using the change of the quantization step size to adapt to the channel code rate, in H.264, the data segmentation method is often used to cope with the change of the channel code rate. In general, the concept of data partitioning is to generate video data with different priorities in the encoder to support quality of service QoS in the network. For example, using the syntax-based seddata partitioning method, each frame of data is divided into several parts according to its importance, which allows discarding less important information when the buffer overflows. A similar temporal data partitioning method can also be employed, which is accomplished by using multiple reference frames in P and B frames. In wireless communication applications, we can support large bit rate changes in wireless channels by changing the quantization accuracy or spatial/temporal resolution of each frame. However, in the case of multicast, it is impossible to require the encoder to respond to varying bit rates. Therefore, unlike the method of Fine Granular Scalability (FGS) used in MPEG-4 (lower efficiency), H.264 uses stream-switched SP frames instead of hierarchical coding. H.264 performance test TML-8 is the test mode of H.264, which is used to compare and test the video coding efficiency of H.264. The PSNR provided by the test results clearly shows that the H.264 results have obvious advantages over the performance of MPEG-4 (ASP: Advanced SimpleProfile) and H.263++ (HLP: HighLatencyProfile), as shown in Figure 5. Show. The PSNR of H.264 is significantly better than MPEG-4 (ASP) and H.263++ (HLP). In the comparison test of 6 rates, the PSNR of H.264 is 2dB higher than MPEG-4 (ASP). It is 3dB higher than H.263 (HLP) on average. The six test rates and their associated conditions are: 32kbit/s rate, 10f/s frame rate and QCIF format; 64kbit/s rate, 15f/s frame rate and QCIF format; 128kbit/s rate, 15f/s frame rate And CIF format; 256 kbit/s rate, 15 f/s frame rate and QCIF format; 512 kbit/s rate, 30 f/s frame rate and CIF format; 1024 kbit/s rate, 30 f/s frame rate and CIF format.