Representation learning is a collection of methods for learning feature techniques in machine learning,and deep learning methods are representation learning methods with multiple representation combinations,which can transform the original input into higher and more advanced through simple linear combination and mapping into nonlinear modules.Multiple levels of abstraction representation,by combining enough transformations of this class,it is possible to learn very complex transformation functions from large datasets.These methods have greatly advanced the state-of-the-art in speech recognition,visual object recognition,object detection,and many other fields such as drug discovery and genomics.Deep learning methods use back-propagation algorithms to instruct machines how to find complex structures in large data sets and change its internal parameters,where deep convolutional networks are widely used to process images,video,speech and audio,while recurrent networks are used for sequential data such as text and speech provide processing solutions,and the latest transformer-like network structure introduces an attention mechanism,which has better performance of mining data features,and gradually replaces deep convolutional network and recurrent network structure,but it requires a larger data-driven,limited application scope.This article explores the development history of deep learning and its applications in different fields,and discusses the development direction of deep machine learning.
Key words
deep learning /
model ecology /
Transformer
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
References
[1] Hinton G E,Osindero S,Teh Y W.Deep Machine LearningA New Frontier in Artificial Intelligence Research[J].A fast learning algorithm for deep belief nets. Neural Computation,2006,18(7):15271554.
[2] Bengio Y,Courville A,Vincent P.Representation learning: A review and new perspectives[J].IEEE transactions on pattern analysis and machine intelligence,2013,35(8):17981828.
[3] Goodfellow I,Bengio Y,Courville A.Deep learning[M].MIT press,2016.
[4] 周飞燕,金林鹏,董军.卷积神经网络研究综述[J].计算机学报,2017,40(6):12291251.
[5] 孙志军,薛磊,许阳明,等.深度学习研究综述[J].计算机应用研究,2012,29(8):28062810.
[6] 郑远攀,李广阳,李晔.深度学习在图像识别中的应用研究综述[J].计算机工程与应用,2019,55(12):2036.
[7] Pearson K LIII.On lines and planes of closest fit to systems of points in space[J].The London,Edinburgh,and Dublin philosophical magazine and journal of science,1901,2(11): 559572.
[8] Fisher R A.The use of multiple measurements in taxonomic problems[J].Annals of eugenics,1936,7(2):179188.
[9] Hebb D O.The organisation of behaviour:a neuropsychologicaltheory[M].New York:Science Editions,1949.
[10] Rosenblatt F.The perceptron:a probabilistic model for information storage and organization in the brain[J].Psychological review,1958,65(6):386.
[11] Werbos P.Beyond regression:new tools for prediction and analysis in the behavioral sciences[D].Ph. D. dissertation,Harvard University,1974.
[12] Hinton G E,Osindero S,Teh Y W.A fast learning algorithm for deep belief nets[J].Neural computation,2006,18(7):15271554.
[13] Bengio Y,Courville A,Vincent P.Representation learning:A review and new perspectives[J].IEEE transactions on pattern analysis and machine intelligence,2013,35(8):17981828.
[14] Goodfellow I,Bengio Y,Courville A.Deep learning[M].MIT press,2016.
[15] Schölkopf B,Locatello F,Bauer S,et al.Toward causal representation learning[J].Proceedings of the IEEE,2021,109(5):612634.
[16] Wang T,Huang J,Zhang H,et al.Visual commonsense rcnn[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:1076010770.
[17] Qi J,Niu Y,Huang J,et al.Two causal principles for improving visual dialog[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:1086010869.
[18] Krizhevsky A,Sutskever I,Hinton G E.Imagenet classification with deep convolutional neural networks[J].Advances in neural information processing systems,2012(25):10971105.
[19] Le Cun Y,Bengio Y.Convolutional networks for images,speech,and time series[J].The handbook of brain theory and neural networks,1995,3361(10):1995.
[20] He K,Zhang X,Ren S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition,2016:770778.
[21] Schuster M,Paliwal K K.Bidirectional recurrent neural networks[J].IEEE transactions on Signal Processing,1997,45(11):26732681.
[22] Hochreiter S,Schmidhuber J.Long shortterm memory[J].Neural computation,1997,9(8):17351780.
[23] Xingjian S H I,Chen Z,Wang H,et al.Convolutional LSTM network: A machine learning approach for precipitation nowcasting[C]//Advances in neural information processing systems,2015:802810.
[24] Vaswani A,Shazeer N,Parmar N,et al.Attention is all you need[C]//Advances in neural information processing systems,2017:59986008.
[25] Girshick R,Donahue J,Darrell T,et al.Rich feature hierarchies foraccurate object detection and semantic segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition,2014:580587.
[26] Girshick R.Fast rcnn[C]//Proceedings of the IEEE international conference on computer vision,2015:14401448.
[27] Ren S,He K,Girshick R,et al.Faster rcnn:Towards realtime object detection with region proposal networks[J].Advances in neural information processing systems,2015(28):9199.
[28] Redmon J,Divvala S,Girshick R,et al.You only look once:Unified,realtime object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition,2016:779788.
[29] Redmon J,Farhadi A.YOLO9000:better,faster,stronger[C]//Proceedings of the IEEE conference on computer vision and pattern recognition,2017:72637271.
PDF(976 KB)
