UROP Research Mentor Project Submission Portal: Submission #785 | Center for Undergraduate Research and Academic Engagement

Submission information

Submission Number: 785

Submission ID: 14551

Submission UUID: dd2a130f-6ae3-4254-8d6c-1e4f006ce921

Submission URI: /urop-research-mentor-project-submission-portal

Submission Update: /urop-research-mentor-project-submission-portal?token=VgVQ8V7YSRtbtwgazX3GBke0dZUwVAcYIDZKbnJYdqg

Created: Wed, 08/14/2024 - 11:37 PM

Completed: Thu, 08/15/2024 - 12:28 AM

Changed: Thu, 08/29/2024 - 11:31 AM

Remote IP address: 217.180.196.50

Submitted by: Anonymous

Language: English

Is draft: No

Webform: UROP Project Proposal Portal

Submitted to: UROP Research Mentor Project Submission Portal

Research Mentor Information

Primary Research Mentor Name Shangqian Gao

Research Mentor Preferred Pronouns {Empty}

When potential research assistants are reaching out via email, what is your preferred honorific? Dr.

Contact Email (FSU Email if affiliated) sg24bi@fsu.edu

Position Title Faculty

FSU College (if applicable) Arts and Sciences

FSU Department or Non-FSU Organization Affiliation Computer Science

Headshot (optional)

Additional Research Mentor(s)

Research Assistant Supervisor (if different from above) {Empty}

Research Assistant Supervisor Preferred Pronouns {Empty}

Research Assistant Supervisor Preferred Honorific? {Empty}

Contact Email (FSU Email if affiliated) {Empty}

Name of Other Faculty/Collaborator(s) (if applicable) {Empty}

Other Faculty/Collaborator(s) Preferred Pronouns {Empty}

Other Faculty/Collaborator(s) Preferred Honorific? {Empty}

Contact Email (FSU Email if affiliated) {Empty}

Overall Project Details

Title of the Project Improve Efficiency for Large Lanuge Models

Project Keywords Large Language Models, Efficiency

Are you currently looking for research assistants? Yes

Number of Research Assistants Needed 2

Relevant Research Assistant Major(s) Computer Science, Electrical Engineering

Project Location: On FSU Main Campus

If the project location is off campus, does the research assistant(s) need to provide their own transportation? {Empty}

Please select the choice that most accurately describes your expectations for the research assistant(s): Partially Remote

Approximately how many hours a week would the research assistant(s) need to work? 10

Roughly what time frame do you expect research assistant(s) to work? Flexible schedule (Combination of business and outside of business. TBD between student and research mentor.)

Overall Research Project Description Large Language Models (LLMs) have gained significant popularity recently. However, their model size is often too large to be deployed on commercial-grade hardware. The objective of this research project is to explore cutting-edge techniques for reducing the size of LLMs, such as weight pruning, structural pruning, and other similar methods. The project begins with the implementation of existing techniques on various LLMs, including OPT, Phi, LLama, and others. With a thorough understanding of the limitations of current methods, novel approaches can be proposed to address these limitations.

Research Tasks Research Tasks:
a. Literature Review on Large Language Models and Model Compression
b. Implement previous model compression methods for Large Language Models
c. Improve the previous model compression algorithms based on the understanding of the Implementation.

Skills that research assistant(s) may need: Programming skills in Python are required.
Knowledge of Linear Algebra and Probability are required.
Experience with Pytorch is highly recommended.

Mentoring Philosophy My mentoring philosophy is built on collaboration, growth, and mutual respect. My role is to guide students in discovering their strengths, overcoming challenges, and achieving their goals. I aim to equip students with foundational knowledge in machine learning, coding, and paper reading relevant to my research. Recognizing that research is a challenging journey, I encourage students to view obstacles as opportunities for learning and growth. By sharing my experiences and offering constructive feedback, I strive to deepen their understanding of key topics. Given the inherent uncertainties in exploration, I am committed to providing students with hands-on experience in implementation, learning, and the development of new ideas, helping them navigate their research with confidence.

Please provide a link to your publications, a video clip, or a website for your research project (if applicable): https://gaosh.github.io/publications/

Please add any additional information here (if applicable): {Empty}

Are you interested in participating in the UROP Research Mentor Roundtable? No

Roundtable times and Zoom links {Empty}

UROP Program Elements

Mentor Handbook, FAQs, and Communication Yes

UROP Performance Evaluation Yes

Materials Grant Yes

UROP Poster Presentation Yes

{Empty}

Year 2024

update url https://cre.fsu.edu/urop-research-mentor-project-submission-portal?element_parents=elements/research_mentor_information/headshot_optional_&ajax_form=1&_wrapper_format=drupal_ajax&token=VgVQ8V7YSRtbtwgazX3GBke0dZUwVAcYIDZKbnJYdqg