publications

2024

1. ICML

   BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model

   Chenwei Xu*, Yu-Chao Huang*, Jerry Yao-Chieh Hu*, and 4 more authors

   International Conference on Machine Learning (ICML), 2024

   * These authors contributed equally to this work

   Abs arXiv Bib Code

   We introduce the Bi-Directional Sparse Hopfield Network (BiSHop), a novel end-to-end framework for deep tabular learning. BiSHop handles the two major challenges of deep tabular learning: non-rotationally invariant data structure and feature sparsity in tabular data. Our key motivation comes from the recent established connection between associative memory and attention mechanisms. Consequently, BiSHop uses a dual-component approach, sequentially processing data both column-wise and row-wise through two interconnected directional learning modules. Computationally, these modules house layers of generalized sparse modern Hopfield layers, a sparse extension of the modern Hopfield model with adaptable sparsity. Methodologically, BiSHop facilitates multi-scale representation learning, capturing both intra-feature and inter-feature interactions, with adaptive sparsity at each scale. Empirically, through experiments on diverse real-world datasets, we demonstrate that BiSHop surpasses current SOTA methods with significantly less HPO runs, marking it a robust solution for deep tabular learning.

   @article{xu2024bishop,
     title = {BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model},
     author = {Xu*, Chenwei and Huang*, Yu-Chao and Hu*, Jerry Yao-Chieh and Li, Weijian and Gilani, Ammar and Goan, Hsi-Sheng and Liu, Han},
     note = {* These authors contributed equally to this work},
     journal = {International Conference on Machine Learning (ICML)},
     year = {2024},
   }

2. arXiv

   Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization

   Yu-Min Tseng*, Yu-Chao Huang*, Teng-Yun Hsiao*, and 4 more authors

   arXiv preprint arXiv:2406.01171, 2024

   * These authors contributed equally to this work

   Abs arXiv Bib Code

   Recently, methods investigating how to adapt large language models (LLMs) for specific sce- narios have gained great attention. Particularly, the concept of persona, originally adopted in dialogue literature, has re-surged as a promising avenue. However, the growing research on persona is relatively disorganized, lacking a systematic overview. To close the gap, we present a comprehensive survey to categorize the current state of the field. We identify two lines of research, namely (1) LLM Role-Playing, where personas are assigned to LLMs, and (2) LLM Personalization, where LLMs take care of user personas. To the best of our knowledge, we present the first survey tailored for LLM role-playing and LLM personalization under the uni- fied view of persona, including taxonomy, current challenges, and potential directions. To foster future endeavors, we actively maintain a paper collection available to the community.

   @article{tseng2024two,
     title = {Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization},
     author = {Tseng*, Yu-Min and Huang*, Yu-Chao and Hsiao*, Teng-Yun and Hsu, Yu-Ching and Foo, Jia-Yin and Huang, Chao-Wei and Chen, Yun-Nung},
     note = {* These authors contributed equally to this work},
     journal = {arXiv preprint arXiv:2406.01171},
     year = {2024},
   }

3. arXiv

   L2O-g†: Learning to Optimize Parameterized Quantum Circuits with Fubini-Study Metric Tensor

   Yu-Chao Huang, and Hsi-Sheng Goan

   arXiv preprint arXiv:2407.14761, 2024

   Abs arXiv Bib Code

   Before the advent of fault-tolerant quantum computers, variational quantum algorithms (VQAs) play a crucial role in noisy intermediate-scale quantum (NISQ) machines. Conventionally, the op- timization of VQAs predominantly relies on manually designed optimizers. However, learning to optimize (L2O) demonstrates impressive performance by training small neural networks to replace handcrafted optimizers. In our work, we propose L2O-g†, a quantum-aware learned optimizer that leverages the Fubini-Study metric tensor (g†) and long short-term memory networks. We theoreti- cally derive the update equation inspired by the lookahead optimizer and incorporate the quantum geometry of the optimization landscape in the learned optimizer to balance fast convergence and gen- eralization. Empirically, we conduct comprehensive experiments across a range of VQA problems. Our results demonstrate that L2O-g† not only outperforms the current SOTA hand-designed opti- mizer without any hyperparameter tuning but also shows strong out-of-distribution generalization compared to previous L2O optimizers. We achieve this by training L2O-g† on just a single generic PQC instance. Our novel quantum-aware learned optimizer, L2O-g†, presents an advancement in addressing the challenges of VQAs, making it a valuable tool in the NISQ era.

   @article{huang2024l2o,
     title = {L2O-g†: Learning to Optimize Parameterized Quantum Circuits with Fubini-Study Metric Tensor},
     author = {Huang, Yu-Chao and Goan, Hsi-Sheng},
     journal = {arXiv preprint arXiv:2407.14761},
     year = {2024},
   }