表征学习是机器学习中学习特征技术的方法集合,深度学习方法是具有多个表示组合的表征学习方法,其通过简单的线性组合和映入非线性模块,能够将原始输入转换为更高、更多的抽象表示层次,通过组合足够多的该类转换,能够从大数据集中学习非常复杂的转换函数。这些方法极大地提高了语音识别、视觉对象识别、对象检测和许多其他领域(如药物发现和基因组学)的最新技术水平。深度学习方法通过使用反向传播算法来指示机器如何在大数据集中发现复杂的结构并改变它的内部参数,其中深度卷积网络广泛应用于图像、视频、语音和音频处理,循环网络则对诸如文本和语音等顺序数据提供处理方案,而最新的Transformer类网络结构引入注意力机制,具有更加良好的挖掘数据特征的性能,逐渐替代深度卷积网络和循环网络结构,但其需要更加庞大的数据驱动,应用范围受限。本文探究深度学习发展历史及其在不同领域的应用,讨论深度机器学习的发展方向。
Abstract
Representation learning is a collection of methods for learning feature techniques in machine learning,and deep learning methods are representation learning methods with multiple representation combinations,which can transform the original input into higher and more advanced through simple linear combination and mapping into nonlinear modules.Multiple levels of abstraction representation,by combining enough transformations of this class,it is possible to learn very complex transformation functions from large datasets.These methods have greatly advanced the state-of-the-art in speech recognition,visual object recognition,object detection,and many other fields such as drug discovery and genomics.Deep learning methods use back-propagation algorithms to instruct machines how to find complex structures in large data sets and change its internal parameters,where deep convolutional networks are widely used to process images,video,speech and audio,while recurrent networks are used for sequential data such as text and speech provide processing solutions,and the latest transformer-like network structure introduces an attention mechanism,which has better performance of mining data features,and gradually replaces deep convolutional network and recurrent network structure,but it requires a larger data-driven,limited application scope.This article explores the development history of deep learning and its applications in different fields,and discusses the development direction of deep machine learning.
关键词
深度学习 /
模型生态 /
Transformer
Key words
deep learning /
model ecology /
Transformer
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Hinton G E,Osindero S,Teh Y W.Deep Machine LearningA New Frontier in Artificial Intelligence Research[J].A fast learning algorithm for deep belief nets. Neural Computation,2006,18(7):15271554.
[2] Bengio Y,Courville A,Vincent P.Representation learning: A review and new perspectives[J].IEEE transactions on pattern analysis and machine intelligence,2013,35(8):17981828.
[3] Goodfellow I,Bengio Y,Courville A.Deep learning[M].MIT press,2016.
[4] 周飞燕,金林鹏,董军.卷积神经网络研究综述[J].计算机学报,2017,40(6):12291251.
[5] 孙志军,薛磊,许阳明,等.深度学习研究综述[J].计算机应用研究,2012,29(8):28062810.
[6] 郑远攀,李广阳,李晔.深度学习在图像识别中的应用研究综述[J].计算机工程与应用,2019,55(12):2036.
[7] Pearson K LIII.On lines and planes of closest fit to systems of points in space[J].The London,Edinburgh,and Dublin philosophical magazine and journal of science,1901,2(11): 559572.
[8] Fisher R A.The use of multiple measurements in taxonomic problems[J].Annals of eugenics,1936,7(2):179188.
[9] Hebb D O.The organisation of behaviour:a neuropsychologicaltheory[M].New York:Science Editions,1949.
[10] Rosenblatt F.The perceptron:a probabilistic model for information storage and organization in the brain[J].Psychological review,1958,65(6):386.
[11] Werbos P.Beyond regression:new tools for prediction and analysis in the behavioral sciences[D].Ph. D. dissertation,Harvard University,1974.
[12] Hinton G E,Osindero S,Teh Y W.A fast learning algorithm for deep belief nets[J].Neural computation,2006,18(7):15271554.
[13] Bengio Y,Courville A,Vincent P.Representation learning:A review and new perspectives[J].IEEE transactions on pattern analysis and machine intelligence,2013,35(8):17981828.
[14] Goodfellow I,Bengio Y,Courville A.Deep learning[M].MIT press,2016.
[15] Schölkopf B,Locatello F,Bauer S,et al.Toward causal representation learning[J].Proceedings of the IEEE,2021,109(5):612634.
[16] Wang T,Huang J,Zhang H,et al.Visual commonsense rcnn[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:1076010770.
[17] Qi J,Niu Y,Huang J,et al.Two causal principles for improving visual dialog[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:1086010869.
[18] Krizhevsky A,Sutskever I,Hinton G E.Imagenet classification with deep convolutional neural networks[J].Advances in neural information processing systems,2012(25):10971105.
[19] Le Cun Y,Bengio Y.Convolutional networks for images,speech,and time series[J].The handbook of brain theory and neural networks,1995,3361(10):1995.
[20] He K,Zhang X,Ren S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition,2016:770778.
[21] Schuster M,Paliwal K K.Bidirectional recurrent neural networks[J].IEEE transactions on Signal Processing,1997,45(11):26732681.
[22] Hochreiter S,Schmidhuber J.Long shortterm memory[J].Neural computation,1997,9(8):17351780.
[23] Xingjian S H I,Chen Z,Wang H,et al.Convolutional LSTM network: A machine learning approach for precipitation nowcasting[C]//Advances in neural information processing systems,2015:802810.
[24] Vaswani A,Shazeer N,Parmar N,et al.Attention is all you need[C]//Advances in neural information processing systems,2017:59986008.
[25] Girshick R,Donahue J,Darrell T,et al.Rich feature hierarchies foraccurate object detection and semantic segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition,2014:580587.
[26] Girshick R.Fast rcnn[C]//Proceedings of the IEEE international conference on computer vision,2015:14401448.
[27] Ren S,He K,Girshick R,et al.Faster rcnn:Towards realtime object detection with region proposal networks[J].Advances in neural information processing systems,2015(28):9199.
[28] Redmon J,Divvala S,Girshick R,et al.You only look once:Unified,realtime object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition,2016:779788.
[29] Redmon J,Farhadi A.YOLO9000:better,faster,stronger[C]//Proceedings of the IEEE conference on computer vision and pattern recognition,2017:72637271.
基金
*国家自然科学基金(51305472);重庆市研究生联合培养基地(JDLHPYJD2018003)。
PDF(976 KB)
