UROP Research Mentor Project Submission Portal: Submission #785

Submission information
Submission Number: 785
Submission ID: 14551
Submission UUID: dd2a130f-6ae3-4254-8d6c-1e4f006ce921

Created: Wed, 08/14/2024 - 11:37 PM
Completed: Thu, 08/15/2024 - 12:28 AM
Changed: Thu, 08/29/2024 - 11:31 AM

Remote IP address: 217.180.196.50
Submitted by: Anonymous
Language: English

Is draft: No
Primary Research Mentor Name Shangqian Gao
Research Mentor Preferred Pronouns
When potential research assistants are reaching out via email, what is your preferred honorific? Dr.
Contact Email (FSU Email if affiliated) sg24bi@fsu.edu
Position Title Faculty
FSU College (if applicable) Arts and Sciences
FSU Department or Non-FSU Organization Affiliation Computer Science
Headshot (optional) prof_pic.jpg
Research Assistant Supervisor (if different from above)
Research Assistant Supervisor Preferred Pronouns
Research Assistant Supervisor Preferred Honorific?
Contact Email (FSU Email if affiliated)
Name of Other Faculty/Collaborator(s) (if applicable)
Other Faculty/Collaborator(s) Preferred Pronouns
Other Faculty/Collaborator(s) Preferred Honorific?
Contact Email (FSU Email if affiliated)
Title of the Project Improve Efficiency for Large Lanuge Models
Project Keywords Large Language Models, Efficiency
Are you currently looking for research assistants? Yes
Number of Research Assistants Needed 2
Relevant Research Assistant Major(s) Computer Science, Electrical Engineering
Project Location: On FSU Main Campus
If the project location is off campus, does the research assistant(s) need to provide their own transportation?
Please select the choice that most accurately describes your expectations for the research assistant(s): Partially Remote
Approximately how many hours a week would the research assistant(s) need to work? 10
Roughly what time frame do you expect research assistant(s) to work? Flexible schedule (Combination of business and outside of business. TBD between student and research mentor.)
Overall Research Project Description Large Language Models (LLMs) have gained significant popularity recently. However, their model size is often too large to be deployed on commercial-grade hardware. The objective of this research project is to explore cutting-edge techniques for reducing the size of LLMs, such as weight pruning, structural pruning, and other similar methods. The project begins with the implementation of existing techniques on various LLMs, including OPT, Phi, LLama, and others. With a thorough understanding of the limitations of current methods, novel approaches can be proposed to address these limitations.
Research Tasks Research Tasks:
a. Literature Review on Large Language Models and Model Compression
b. Implement previous model compression methods for Large Language Models
c. Improve the previous model compression algorithms based on the understanding of the Implementation.
Skills that research assistant(s) may need: Programming skills in Python are required.
Knowledge of Linear Algebra and Probability are required.
Experience with Pytorch is highly recommended.
Mentoring Philosophy My mentoring philosophy is built on collaboration, growth, and mutual respect. My role is to guide students in discovering their strengths, overcoming challenges, and achieving their goals. I aim to equip students with foundational knowledge in machine learning, coding, and paper reading relevant to my research. Recognizing that research is a challenging journey, I encourage students to view obstacles as opportunities for learning and growth. By sharing my experiences and offering constructive feedback, I strive to deepen their understanding of key topics. Given the inherent uncertainties in exploration, I am committed to providing students with hands-on experience in implementation, learning, and the development of new ideas, helping them navigate their research with confidence.
Please provide a link to your publications, a video clip, or a website for your research project (if applicable): https://gaosh.github.io/publications/
Please add any additional information here (if applicable):
Are you interested in participating in the UROP Research Mentor Roundtable? No
Roundtable times and Zoom links
Mentor Handbook, FAQs, and Communication Yes
UROP Performance Evaluation Yes
Materials Grant Yes
UROP Poster Presentation Yes
Year 2024
update url https://cre.fsu.edu/urop-research-mentor-project-submission-portal?element_parents=elements/research_mentor_information/headshot_optional_&ajax_form=1&_wrapper_format=drupal_ajax&token=VgVQ8V7YSRtbtwgazX3GBke0dZUwVAcYIDZKbnJYdqg