Self-Supervised Learning has become an exciting direction in AI community.
Jitendra Malik: 'Supervision is the opium of the AI researcher'
Alexei A. Efros: 'The AI revolution will not be supervised'
Yann LeCun: 'self-supervised learning is the cake, supervised learning is the icing on the cake, reinforcement learning is the cherry on the cake'
本综述来自西湖大学人工智能研究与创新中心(Center for AI Research and Innovation,Westlake University),对现有的图自监督学习技术进行了全面的回顾。实验室目前研究方向包括:图自监督与半监督学习,图结构优化,时序动态图,及其在生命科学等交叉领域的应用。欢迎优秀的同学申请实习访问,及优秀的博后加入我们。完整的综述请参照原文:
[1] Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang, and Y. Shen, “Graph contrastive learning with augmentations,”Advances in Neural Information Processing Systems, vol. 33, 2020.
[2] J. Qiu, Q. Chen, Y. Dong, J. Zhang, H. Yang, M. Ding, K. Wang, and J. Tang, “Gcc: Graph contrastive coding for graph neural network pre-training,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020, pp. 1150–1160.
[3] S. Thakoor, C. Tallec, M. G. Azar, R. Munos, P. Veliˇckovi´c, and M. Valko “Bootstrapped representation learning on graphs,” arXiv preprint arXiv:2102.06514, 2021.
[4] P. Velickovic, W. Fedus, W. L. Hamilton, P. Li` o, Y. Bengio, and R. D. Hjelm, “Deep graph infomax.” in ICLR (Poster), 2019.
[5] Y. Jiao, Y. Xiong, J. Zhang, Y. Zhang, T. Zhang, and Y. Zhu, “Sub-graph contrast for scalable selfsupervised graph representation learning,” arXiv preprint arXiv:2009.10273, 2020.
[6] F.-Y. Sun, J. Hoffmann, V. Verma, and J. Tang, “Infograph: Unsupervised and semi-supervised graphlevel representation learning via mutual information maximization,” arXiv preprint arXiv:1908.01000, 2019.
[7] F. Manessi and A. Rozza, “Graph-based neural network models with multiple self-supervised auxiliary tasks,” arXiv preprint arXiv:2011.07267, 2020.
[8] Z. Hu, Y. Dong, K. Wang, K.-W. Chang, and Y. Sun,“Gpt-gnn: Generative pre-training of graph neural networks,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020, pp. 1857–1867.
[9] W. Jin, T. Derr, H. Liu, Y. Wang, S. Wang, Z. Liu, and J. Tang, “Self-supervised learning on graphs: Deep insights and new direction,” arXiv preprint arXiv:2006.10141, 2020.
[10] Z. Hu, C. Fan, T. Chen, K.-W. Chang, and Y. Sun, “Pretraining graph neural networks for generic structural feature extraction,” arXiv preprint arXiv:1905.13728, 2019.
[11] Y. Rong, Y. Bian, T. Xu, W. Xie, Y. Wei, W. Huang, and J. Huang, “Self-supervised graph transformer on large-scale molecular data,” Advances in Neural Information Processing Systems, vol. 33, 2020.
[12] S. Zhang, Z. Hu, A. Subramonian, and Y. Sun, “Motifdriven contrastive learning of graph representations,”arXiv preprint arXiv:2012.12533, 2020.
[13] Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang,“Graph contrastive learning with adaptive augmentation,”arXiv preprint arXiv:2010.14945, 2020.
[14] N. Jovanovi´c, Z. Meng, L. Faber, and R. Wattenhofer,“Towards robust graph contrastive learning,” arXiv preprint arXiv:2102.13085, 2021.
[15] Z. Peng, Y. Dong, M. Luo, X.-M. Wu, and Q. Zheng, “Self-supervised graph representation learning via global context prediction,” arXiv preprint arXiv:2003.01604, 2020.
[16] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, “A comprehensive survey on graph neural networks,” IEEE transactions on neural networks and learning systems, 2020.
[17] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019.
[18] K. He, H. Fan, Y.Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9729–9738.
[19] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning. PMLR, 2020, pp. 1597–1607.
[20] F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, H. Xiong, and Q. He, “A comprehensive survey on transfer learning,” Proceedings of the IEEE, vol. 109, no. 1, pp. 43–76, 2020.
[21] J. Zeng and P. Xie, “Contrastive self-supervised learning for graph classification,” arXiv preprint arXiv:2009.05923, 2020.