基于手工特征的视频哈希研究综述

doi:10.16088/j.issn.1001-6600.2022022101

摘要/Abstract

摘要： 视频哈希是从视频中提取到的基于视觉内容的短小数字序列,在实际应用中,用视频哈希来表示视频,能降低视频的存储代价和视频相似计算的复杂度。目前,视频哈希已被广泛应用于拷贝检测、篡改取证、视频索引、视频检索等方面。近年,视频哈希研究取得许多重要进展,研究人员设计和开发出多种手工特征提取技术,并建立一系列视频哈希算法。本文将基于手工特征的视频哈希算法分为空域计算和时空域计算2个大类,其中基于空域计算的哈希算法又分为逐帧计算和关键帧计算2类,而基于时空域计算的哈希算法则分为正交变换、统计特征、视觉特征点、数据降维和其他技术5类。根据这些分类,本文先分析每类算法的代表性研究成果并总结其性能;然后介绍常用的哈希度量方法、性能评价指标和视频数据集;最后列出未来研究工作可重点关注的内容,包括面向篡改取证的视频哈希、基于深度学习的高效视频哈希和面向移动应用的轻量级视频哈希等。

关键词: 视频哈希, 手工特征, 特征提取, 关键帧, 数据降维

Abstract: Video hash is a short digital sequence based on visual content extracted from video. In practical applications, the strategy of representing a video by its video hash can reduce storage cost of video and complexity of calculating video similarity. Video hash is widely used in many applications, such as copy detection, tampering forensics, video indexing, and video retrieval, etc. In recent years, there are many progresses on video hashing research. Researchers have designed and developed some techniques of hand-craft feature extraction and presented many video hashing algorithms. This paper classifies the hand-craft features based video hashing algorithms into two categories: spatial computation, and spatial-temporal computation. Moreover, the video hashing algorithms based on spatial computation are further divided into two sub-categories: frame-by-frame calculation and key frame calculation. The video hashing algorithms based on spatial-temporal computation are further divided into five sub-categories: orthogonal transform, statistical feature, visual feature point, data dimensionality reduction and other technique. In this paper, some typical algorithms of each category are first described, and their performances are then summarized. Next, the common-used metrics of hash similarity, performance evaluation indices and video datasets are introduced. Finally, the future trend of research is presented, including video hashing for tampering forensics, efficient video hashing based on deep learning, lightweight video hashing for mobile applications, and so on.

Key words: video hashing, hand-craft features, feature extraction, key frame, data dimensionality reduction

中图分类号:

TP391.41

于梦竹, 唐振军. 基于手工特征的视频哈希研究综述[J]. 广西师范大学学报（自然科学版）, 2022, 40(5): 72-89.

YU Mengzhu, TANG Zhenjun. Survey of Video Hash Research Based on Hand-craft Features[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(5): 72-89.

参考文献

[1]SCHNEIDER M, CHANG S F. A robust content based digital signature for image authentication[C]// Proceedings of the 3rd IEEE International Conference on Image Processing. Piscataway, NJ: IEEE Press, 1996: 227-230. DOI:10.1109/ICIP.1996.560425.
[2]OOSTVEEN J C, KALKER T, HAITSMA J. Visual hashing of digital video: applications and Techniques[C]// Proceedings of SPIE 4472, Applications of Digital Image Processing XXIV. Bellingham, WA: SPIE, 2001: 121-131. DOI:10.1117/12.449746.
[3]TANG Z J, ZHANG X Q, LI X X, et al. Robust image hashing with ring partition and invariant vector distance[J]. IEEE Transactions on Information Forensics and Security, 2016, 11(1): 200-214. DOI:10.1109/TIFS.2015.2485163.
[4]TANG Z J, CHEN L, ZHANG X Q, et al. Robust image hashing with tensor decomposition[J]. IEEE Transactions on Knowledge and Data Engineering, 2019, 31(3): 549-560. DOI:10.1109/TKDE.2018.2837745.
[5]SONG J K, GAO L L, LIU L, et al. Quantization-based hashing: a general framework for scalable image and video retrieval[J]. Pattern Recognition, 2018, 75:175-187. DOI:10.1016/j.patcog.2017.03.021.
[6]WARY A, NEELIMA A. A review on robust video copy detection[J]. International Journal of Multimedia Information Retrieval, 2019, 8(2): 61-78. DOI:10.1007/s13735-018-0159-x.
[7]MUCEDEROA, LANCINI R, MUPELLI F. A novel hashing algorithm for video sequences[C]// 2004 International Conference on Image Processing (ICIP 2004): Volume 4. Piscataway, NJ: IEEE Press, 2004: 2239-2242. DOI:10.1109/ICIP.2004.1421543.
[8]LEE S, YOO C D. Robust video fingerprinting for content-based video identification[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2008, 18(7): 983-988. DOI:10.1109/TCSVT.2008.920739.
[9]YANG G B, CHEN N, JIANG Q. A robust hashing algorithm based on SURF for video copy detection[J]. Computers & Security, 2012, 31(1): 33-39. DOI:10.1016/j.cose.2011.11.004.
[10]PENG H Y, DENG C, AN L L, et al. Learning to multimodal hash for robust video copy detection[C]// 2013 IEEE International Conference on Image Processing. Piscataway, NJ: IEEE Press, 2013: 4482-4486. DOI:10.1109/ICIP.2013.6738923.
[11]HIMEUR Y, SADI K A. Joint color and texture descriptor using ring decomposition for robust video copy detection in large databases[C]// 2015 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT). Piscataway, NJ: IEEE Press, 2015: 495-500. DOI:10.1109/ISSPIT.2015.7394386.
[12]TANG Z J, ZHANG X Q, ZHANG S C. Robust perceptual image hashing based on ring partition and NMF[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(3): 711-724. DOI:10.1109/TKDE.2013.45.
[13]HIMEUR Y, SADI K A. Robust video copy detection based on ring decomposition based binarized statistical image features and invariant color descriptor (RBSIF-ICD)[J]. Multimedia Tools and Applications, 2018, 77(13): 17309-17331. DOI:10.1007/s11042-017-5307-4.
[14]SUN R, YAN X X, GAO J. Robust video fingerprinting scheme based on contourlet hidden Markov tree model[J]. Optik, 2017, 128: 139-147. DOI:10.1016/j.ijleo.2016.09.105.
[15]CHEN L, YE D P, JIANG S Z. High accuracy perceptual video hashing via low-rank decomposition and DWT[C]// MMM2020: Multimedia Modeling: LNCS Volume 11961. Cham: Springer, 2020: 802-812. DOI:10.1007/978-3-030-37731-1_65.
[16]DE ROOVER C, DE VLEESCHOUWER C, LEFEBVRE F, et al. Robust video hashing based on radial projections of key frames[J]. IEEE Transaction on Signal Processing, 2005, 53(10): 4020-4037. DOI:10.1109/TSP.2005.855414.
[17]NIE X S, QIAO J P, LIU J, et al. LLE-based video hashing for video identification[C]// IEEE 10th International Conference on Signal Processing Proceedings. Piscataway, NJ: IEEE Press, 2010: 1837-1840. DOI:10.1109/ICOSP.2010.5656914.
[18]NIE X S, LIU J, SUN J D, et al. Key-frame based robust video hashing using isometric feature mapping[J]. Journal of Computational Information Systems, 2011, 7(6): 2112-2119.
[19]KIM S, LEE S H, RO Y M. Rotation and flipping robust region binary patterns for video copy detection[J]. Journal of Visual Communication and Image Representation, 2014, 25(2): 373-383. DOI:10.1016/j.jvcir.2013.12.003.
[20]NEELIMA A, SINGH K M. Collusion and rotation resilient video hashing based on scale invariant feature transform[J]. The Image Science Journal, 2017, 65(1): 62-74. DOI:10.1080/13682199.2016.1260216.
[21]COSKUN B, SANKUR B, MEMON N. Spatio-temporal transform based video hashing[J]. IEEE Transactions on Multimedia, 2006, 8(6): 1190-1208. DOI:10.1109/TMM.2006.884614.
[22]LI Y N. Energy based robust video hash algorithm[C]// 2010 International Conference on Computational Intelligence and Security. Los Alamitos, CA: IEEE Computer Society, 2010: 433-436. DOI:10.1109/CIS.2010.100.
[23]ESMAEILI M M, FATOURECHI M, WARD R K. A robust and fast video copy detection system using content-based fingerprinting[J]. IEEE Transactions on Information Forensics and Security, 2011, 6(1): 213-226. DOI:10.1109/TIFS. 2010.2097593.
[24]SETYAWAN I, TIMOTIUS I K. Spatio-temporal digital video hashing using edge orientation histogram and discrete cosine transform[C]// 2014 International Conference on Information Technology Systems and Innovation (ICITSI). Piscataway, NJ: IEEE Press, 2014: 111-115. DOI:10.1109/ICITSI.2014.7048247.
[25]TANG W, WO Y, HAN G Q. Geometrically robust video hashing based on ST-PCT for video copy detection[J]. Multimedia Tools and Applications, 2019, 78(15): 21999-22022. DOI:10.1007/s11042-019-7513-8.
[26]KHELIFI F, BOURIDANE A. Perceptual video hashing for content identification and authentication[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 29(1): 50-67. DOI:10.1109/TCSVT.2017.2776159.
[27]TANG Z J, CHEN L, YAO H, et al. Video hashing with DCT and NMF[J]. The Computer Journal, 2020, 63(7): 1017-1030. DOI:10.1093/comjnl/bxz060.
[28]ZHOU X B, SCHMUCKER M, BROWN C L. Perceptual hashing of video content based on differential block similarity[C]// Computational Intelligence and Security: LNAI Volume 3802. Berlin: Springer, 2005: 80-85. DOI:10.1007/11596981_12.
[29]聂秀山, 乔建苹, 秦丰林. 基于拉普拉斯特征映射的鲁棒视频哈希方法[J]. 计算机工程与设计, 2011, 32(11): 3799-3802. DOI:10.16208/j.issn1000-7024.2011.11.053.
[30]XIANG S J, YANG J Q, HUANG J W. Perceptual video hashing robust against geometric distortions[J]. Science China: Information Sciences, 2012, 55(7): 1520-1527. DOI:10.1007/s11432-011-4450-1.
[31]朱映映, 文振焜, 杜以华, 等. 基于视频感知哈希的视频篡改检测与多粒度定位[J]. 中国图象图形学报, 2013, 18(8): 924-932. DOI:10.11834/jig.20130806.
[32]文振焜, 高金花, 朱映映, 等. 融合时空域变化信息的视频感知哈希算法研究[J]. 电子学报, 2014, 42(6): 1163-1167. DOI:10.3969/j.issn.0372-2112.2014.06.019.
[33]TANG Z J, ZHANG S P, ZHANG X Q, et al. Video hashing with secondary frames and invariant moments[J]. Journal of Visual Communication and Image Representation, 2021, 79: 103209. DOI:10.1016/j.jvcir.2021.103209.
[34]ZHAO Y X, LIU G J, DAI Y W, et al. Robust hashing based on persistent points for video copy detection[C]// 2008 International Conference on Computational Intelligence and Security. Los Alamitos, CA: IEEE Computer Society, 2008: 305-308. DOI:10.1109/CIS.2008.175.
[35]LI Y N, LU Z M. Video identification using spatio-temporal salient points[C]// 2009 Fifth International Conference on Information Assurance and Security. Los Alamitos, CA: IEEE Computer Society, 2009: 79-82. DOI:10.1109/IAS.2009. 291.
[36]VRETOS N, NIKOLAIDIS N, PITAS I. Video fingerprinting using Latent Dirichlet Allocation and facial images[J]. Pattern Recognition, 2012, 45(7): 2489-2498. DOI:10.1016/j.patcog.2011.12.022.
[37]魏晖, 杨高波, 夏明. 一种基于取证哈希的数字视频篡改取证方法[J]. 电子与信息学报, 2013, 35(12): 2934-2941.
[38]NIE X S, LIU J, SUN J D, et al. Robust video hashing based on double-layer embedding[J]. IEEE Signal Processing Letters, 2011, 18(5): 307-310. DOI:10.1109/LSP.2011.2126020.
[39]TANG Z J, ZHANG S P, CHEN Z H, et al. Robust video hashing based on multidimensional scaling and ordinal measures[J]. Security and Communication Networks, 2021, 2021: 9930673. DOI:10.1155/2021/9930673.
[40]LI M, MONGA V. Robust video hashing via multilinear subspace projections[J]. IEEE Transactions on Image Processing, 2012, 21(10): 4397-4409. DOI:10.1109/TIP.2012.2206036.
[41]NIE X S, YIN Y L, SUN J D, et al. Comprehensive feature-based robust video fingerprinting using tensor model[J]. IEEE Transactions on Multimedia, 2017, 19(4): 785-796. DOI:10.1109/TMM.2016.2629758.
[42]文振焜, 高金花, 杜以华, 等. 鲁棒可区分的压缩视频感知哈希算法研究[J]. 深圳大学学报(理工版), 2013, 30(2): 157-161. DOI:10.3724/SP.J.1249.2013.02157.
[43]NIE X S, CHAI Y N, LIU J, et al. Spherical torus-based video hashing for near-duplicate video detection[J]. Science China: Information Sciences, 2016, 59(5): 059101. DOI:10.1007/s11432-016-5528-6.
[44]CHEN Z H, TANG Z J, ZHANG X P, et al. Efficient video hashing based on low-rank frames[J]. IET Image Processing, 2022, 16(2): 344-355. DOI:10.1049/ipr2.12351.
[45]欧阳杰, 高金花, 文振焜, 等. 融合HVS计算模型的视频感知哈希算法研究[J]. 中国图象图形学报, 2011, 16(10): 1883-1889. DOI:10.11834/jig.20111005.
[46]LIU X C, SUN J D, LIU J. Visual attention based temporally weighting method for video hashing[J]. IEEE Signal Processing Letters, 2013, 20(12): 1253-1256. DOI:10.1109/LSP.2013.2287006.
[47]SUN J D, LIU X C, WAN W B, et al. Video hashing based on appearance and attention features fusion via DBN[J]. Neurocomputing, 2016, 213: 84-94. DOI:10.1016/j.neucom.2016.05.098.
[48]LI M, MONGA V. Twofold video hashing with automatic synchronization[J]. IEEE Transactions on Information Forensics and Security, 2015, 10(8): 1727-1738. DOI:10.1109/TIFS.2015.2425362.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed