Evaluation of STEM Engagement Activities on the Attitudes and Perceptions of Mechanical Engineering S-STEM Scholars

The science, technology, engineering, and mathematics (STEM) industries in the United States face a looming retirement cliff as skilled baby boomers begin to retire and leave the workforce [1]. The continuous increase in STEM job openings requires highly qualified STEM professionals to fill those positions to maintain the competitive edge of the country. Improvements in student retention and graduation rates, while providing students with a robust college STEM education, are essential to meet the demands of regional and national employers.

Sponsored by the National Science Foundation (NSF) since 2009, the mechanical engineering (ME) S-STEM scholarship program at the University of Maryland Baltimore County (UMBC) was established to provide enhanced educational opportunities to undergraduate students in the ME department. Our program uses effective strategies suggested in previous studies to address students' psychosocial needs and to enhance retention and graduation. Prior research has identified several psychosocial variables that drive students to stay or leave STEM [2–4]. These variables include self-efficacy and outcome expectations, science identity, sense of belonging, and academic integration. Addressing these variables is critical for developing effective STEM education programs.

Self-efficacy is defined as the personal judgment of one's capabilities to execute behavior to attain certain goals. Outcome expectations are anticipated consequences of one's actions. According to the social cognitive theory [5,6], individuals' self-efficacy beliefs and outcome expectations are shaped by their experiences and strongly determine their behavior. The social cognitive career theory [7,8] also posits that career interests and choices are determined to a great extent by one's self-efficacy and environmental supports. Self-efficacy and outcome expectations are known to be predictive of STEM retention, degree completion, and graduate school entry [2,9–12].

Identity refers to a set of meanings that define a person's roles and group memberships. More specifically, science identity refers to the extent to which students see themselves as scientists. The identity theory [13,14] postulates that social structures shape the development of one's identities for different social roles and, when activated in social settings, identities influence behavior. Consistent with the identity theory, previous research has supported the link between science identity and STEM persistence and achievement [2,3].

The theory of psychological sense of community [15] stresses the importance of a sense of belonging, a sense of mattering, and integration within a community. Sense of belonging is defined as a student's psychological sense of identification and affiliation with the campus community [16]. As a behavioral aspect of belonging, academic integration refers to the extent to which students adapt to the academic way of life on campus [17]. Sense of belonging and academic integration are strong predictors of STEM achievement and persistence [18,19].

Given these theoretical and empirical support from prior research, our ME S-STEM program was focused on providing diverse activities and opportunities to boost scholars' self-efficacy and outcome expectations, shape their science/engineering identities, and build a sense of community among scholars to enhance retention and graduation. Two major featured activities of our program are mentoring and research experience.

Mentoring is an essential catalyst for fostering academic success [20,21] and is especially important for women and students who are traditionally underrepresented in STEM fields [22,23]. Effective faculty mentoring is associated with improved academic performance, higher rates of retention, positive identity development, and better integration into the campus [24–27]. In addition to mentoring provided by faculty, peer mentors can draw on their recent and relatable experiences to provide resources, guidance, and support to students. Peer mentoring also contributes to students' integration into the campus, increases retention, and enhances achievement [28].

Research experience is another effective strategy for fostering academic success. It is critical for developing and sustaining a student's interests in STEM careers. Involvement in research is associated with high rates of retention and graduation, a high likelihood of graduate school matriculation, increased self-efficacy, and a greater sense of belonging to the campus [29–32]. Therefore, research experience has become an integral component of STEM education across higher education institutions [33].

In the ME S-STEM scholarship program, the selected students receive scholarships and supplemental program activities to facilitate their full-time enrollment to improve scholastic achievement, with the goal to lead to industrial job placements and/or graduate school enrollments. In addition to financial support, our S-STEM scholars are connected to individual faculty mentors and are provided with opportunities of internships and research experiences. Incorporating research experiences into the education curriculum is a strength of our program [34–36]. Specific activities include research seminars emphasizing bio-engineering, in-depth laboratory visits, featuring scholars' research on our website, sponsoring scholars' conference attendance, and offering research opportunities in the summer or academic year. The research experiences have helped attract more female and minority students to mechanical engineering, expand scholars' skill base, and provide successful paths for graduate study. Another successful component of the program is community building tailored for commuting students, including an annual retreat, lunch with faculty members, and workshops providing academic and professional development support.

Since its inception in 2009, the program has supported more than 110 undergraduate students with diverse ethical and economic backgrounds. We have achieved a retention rate of 89% in the ME S-STEM Scholarship program [36]. Among the 75 ME S-STEM scholars who graduated, 32% are pursuing graduate degrees in a STEM major, 64% are now working in an STEM industry, and 4% are looking for a job opportunity. The high retention rate and percentage of students pursuing graduate school suggest positive impacts of our program activities on our scholars. In the meantime, we have observed a steady increase in the total enrollment in ME department undergraduates. The student demographics in the ME undergraduate program become more diverse, with significant increases in the percentages of community college transfer students, female, and Hispanic students in the undergraduate student population [36]. It is important to identify cost-effective practices sponsored by our program so that the best practices can be disseminated to other engineering programs in the country. It would lead to determination of activities that require minimal resources but can be easily implemented to improve retention and graduation rates of undergraduate students in mechanical engineering.

In this study, we present data that reveal the attitudes and perceptions of the current ME S-STEM scholars based on surveys conducted in 2019 and 2020. The survey was designed to measure their attitudes and perspectives on science/engineering identity, research efficacy, expectation, integration, and sense of belonging. The scholars were then asked to provide their opinion on the impact that the activities sponsored by our S-STEM program had on their attitudes and perceptions. The goal of this study was to identify program activities that were most effective to help students for improvements.

A survey was developed by our research team containing measurable items regarding student attitudes, perspectives, science/engineering identity, and research self-efficacy. The survey questions were entered in Qualtrics^XM, and the students were invited via emails to participate in the online survey. The survey was anonymous and voluntary, although we encouraged the scholars in the program to take the survey. No identifiable information of the participants was collected from the survey, thus, choosing not to participate will neither affect students' grades in ME courses nor their scholarship renewals. Once students decided to take part, they were free to withdraw at any time. The research protocol for collecting and analyzing survey data in this study was submitted to the Institutional Review Board (IRB) at UMBC, and the protocol was approved on Nov. 9, 2017.

The first section of the survey consisted of 10 questions focusing on students' demographic information including race, gender, family economic status, GPAs, whether they originally transferred from a community college, etc. The second section contained Likert-scaled items to include Research Self-Efficacy (nine questions), Science/Engineering Identity (five questions), Expectations and Goals (four questions), Academic Integration (five questions), and Senses of Belonging to Program and Campus (8 questions). The following describes development of the questions in each category.

Research self-efficacy was measured by items from the scientific self-efficacy scale [2] that assesses a student's ability to function as a scientist/engineer on a scale of one (not at all confident) to five (absolutely confident). Sample items include I can use technical science skills, generate a research question, formulate hypothesis, collect and analyze data, interpret results, and use scientific literature and/or reports to guide research and develop theories.

Scientific/engineering identity questions were designed based on the items from the scientific identity scale [2,37] for students to assess how much being a scientist/engineer contributes to how they identify themselves. Questions measure whether students have a strong sense of belonging to the community of scientists/engineers, feel like they belong in the field of science/engineering, and have come to think of themselves as scientists/engineers on a scale of one (strongly disagree) to five (strongly agree).

Based on the social cognitive theory and the social cognitive career theory [5–8], the research team developed questions in the category of expectations and goals, aimed at measuring the student's confidence in achieving their goals in academia and research. Sample question statements in this category include I will achieve my career goals, I am excited about the idea of scientific research, I will achieve my academic goals, and I am comfortable going to ME faculty and staff if I have a problem. Each question was measured on a scale of one (strongly disagree) to five (strongly agree).

Based on the Your First College Year survey conducted by the Higher Education Research Institute at the University of California Los Angeles [38], we designed questions to measure students' perspectives on their understanding of academic integration in mechanical engineering program. Questions were included to ask students to quantify how well they understand what their professors expect of them academically, adjust to the academic demands of college, manage their time effectively, and develop close friendships with other students. Each question was measured on a scale of one (very difficult) to four (very easy).

The questions in the category of sense of belonging to the program and to campus were adapted from the Sense of Community Index, which was developed based on McMillan and Chavis's theory of psychological sense of belonging [39]. Sample questions, each on a scale of one (very unsatisfied) to five (very satisfied), included I feel a sense of belonging to STEM community/major/department, I can trust people in the program, I expect to be a part of the program for a long time, I feel comfortable on campus, and my college is supportive of me.

After answering the questions in each category, the students were asked to check any activities they felt that contributed positively to their attitude and perspective in each category. In Table 1, the 15 listed activities are placed into one of four groups: mentoring, academic help, research, and community building. Note that all except the regular faculty/staff mentoring are extra activities initiated and/or promoted by the ME S-STEM scholarship program.

Typically, all ME undergraduate students receive advisement by staff members in the Dean's office during the first year. After students satisfy the requirements of four “gateway” courses, they are officially admitted to the ME program, and the students are then assigned a faculty member in the mechanical engineering department as their regular faculty advisor. Undergraduate students may only talk to their regular faculty advisor once during a semester to get clearance on their registration for the following semester. Extra faculty mentoring in Table 1 refers to the mentoring from a faculty mentor assigned to each scholar in our scholarship program. The faculty mentor is different from the student's regular advisor assigned by the department. The ME S-STEM scholars are encouraged to meet their faculty mentors when needed to discuss not only their academic progress but also their future career plan and personal issues. Most of the scholars had talked to their faculty mentors more than once each semester, suggesting that they felt comfortable to discuss issues they had with the faculty. Based on the annual surveys conducted in the past several years, approximately 54% of the scholars had 1–2 meetings, more than 46% had at least three meetings, and 10% had at least five meetings with their mentors each year. Peer mentoring is a departmental program open to all undergraduate students. The ME S-STEM scholars are required to join this program and serve as either a mentor or a mentee. Students in the peer mentoring program meet regularly during semesters.

The ME S-STEM program also provides academic help to our students by organizing workshops on time management and tutoring services to our scholars. The faculty in our program actively encourage our scholars to pursue internship opportunities by connecting them to employers in industry and/or resources on campus. Various research-related activities are implemented in our program, as shown in Table 1. Considering that most of the undergraduate students on our campus are commuters, we implemented many community building activities to promote interaction among the scholars and with their faculty mentors. We conduct our annual retreat and sponsor social events such as lunch with faculty and workshops to create opportunities to promote gathering and interaction. All our scholars are also encouraged to join professional societies to interact with their peers. With the support of the ME Department, a room in our Engineering building is assigned for scholars to use during the daytime when they are on campus.

The survey also included questions to measure student retention and graduation. If the students were not going to graduate from the ME program that semester, they were asked whether they would continue in the ME program. If the students were to graduate after that semester, they were asked whether they planned to work in STEM industries and/or enroll in graduate school. Finally, they were asked to identify any activities listed in Table 1 that positively contributed to their retention or decision after their graduation.

The survey data collected in this study were numerically coded at a scale from 0 to 100, with 100 indicating the most positive level of variables being measured, i.e., Absolutely Confident or Strongly Agree or Very Easy, as shown in Table 2. Any activity identified by student would be counted as 1, and then the percentage of the total number of identifications to the total number of the surveys would be an indication of how this activity positively contributed to individual categories or decisions. The raw data of the surveys from Qualtrics^XM were then entered into EXCEL for analyses.

Twenty-five surveys from ME S-STEM scholars were completed in June 2019, and another eleven surveys were received in June 2020 from new ME S-STEM scholars entering our scholarship program in Fall 2019. A total of 36 surveys were validated and assessed. Approximately 44% percent of the students self-identified as female and 56% as male. More than 36% of the participants were underrepresented minorities (URMs) defined as African Americans, Hispanics, or Native Americans. More than 30% of the students submitted the survey stated that they were first generation college students. 10 of the 36 students or 28% stated that they transferred from a local community college. Students indicating their race as white were the largest percentage of the scholars (56%), while Asian students were the smallest percentage (14%). There were overlaps between the groups in the demographic data, for example, white and Hispanic (8.3%), female and white (19.4%), female and URMs (8.3%), Asian and white (2.7%). Students from all class standings defined by the number of years in school, including freshmen (22%), sophomores (17%), juniors (19%), and seniors (42%) in the ME department, participated in the survey.

Scores in both academic integration and research self-efficacy were below 80 over a scale of 100 (Fig. 1 and Table 2). An average score of 70 in academic integration suggests that on average, students feel academic integration is between somewhat difficult and somewhat easy. The next low average score of 70.8 was in achieving research self-efficacy. Students felt more than moderately confident but less than very confident in achieving research self-efficacy. Students agreed that being an engineer/scientist is part of who they are in science/engineering identity, with a score of 87.2. They also agreed that they understand the expectations and goals (average score = 81.9), and they were satisfied with their senses of belonging to both the campus and program (average score = 87.4). Note that the results in Fig. 1 agree with that in previous studies that reported lower scores also in the categories of academic integration and research self-efficacy [40].

Figure 2 gives the scores of individual mentoring activities that have contributed positively to the five categories. Not surprisingly, regular mentoring by the faculty/staff assigned to the student was viewed by the students with the most positive impact (37%–61%). This is consistent with previous research that faculty mentoring is an effective strategy to enhance learning [20–28]. The other two mentoring activities, i.e., peer mentoring and extra faculty mentoring from the ME S-STEM Scholarship Program were also considered important. Approximately half of the scholars considered peer mentoring as having positive impacts (44%–53%). The extra faculty mentoring provided by the ME S-STEM Scholarship Program had the most impact on the category of helping our scholars in gaining scientific/engineering identity (44%), followed by 37% in the category of helping them understand expectations and goals.

Shown in Fig. 3, in the three activities under “academic help,” having an internship was considered as helping students gain the ability to function as engineers (61%), understand their expectations and goals (42%), and integrate into campus life (24%). For mechanical engineering students, having an internship in industry provides them early exposure and valuable experience in their chosen career, thus demonstrating the application of the academic fundamentals they learn in class room. This is consistent with previous research showing that internships have a positive impact on engineering students [41]. Considering the low average score received in the category of scientific/engineering efficacy, one may conclude that helping students get an internship opportunity in industry would be an effective way to improve students' scientific/engineering efficacy. The time management workshop sponsored by our program was also considered an effective activity (28%) to help them gain scientific/engineering identity. However, extra tutoring provided by our scholarship program appeared to be perceived as less important than the other two activities, with percentages less than 16%. The ME S-STEM Scholarship Program at UMBC hired a graduate student to serve as a tutor to the scholars in the program. It is a costly activity with an annual budget of $5000, and it does not seem to yield as much positive outcome as originally expected.

The ME S-STEM scholarship program has implemented many activities related to research. As shown in Fig. 4, the in-depth lab visits, research seminars, and research experience for undergraduate (REU) opportunities provided to scholars had the most impact, especially in scientific self-efficacy, science/engineering identity and expectations. In-depth lab visits organized by the ME S-STEM Scholarship Program have been perceived by the students as the most effective activity among the four in the group (36%–72%). Providing in-depth research lab tours for undergraduate students exposes students to current research projects in the department and provides them with the opportunity to explore those projects further. This activity was initiated by our program in 2016 due to comments from students that most lab visits then were routinely limited to 15 min, and students often got only a superficial idea of research conducted in faculty labs. Since 2016, in-depth lab visits, with a duration of 75 min, have become an annual event open to all the undergraduate students in the department. The high percentage of the current scholars agrees well with students' overall satisfaction with the lab visits [36]. Further, students in the survey indicated that the mandatory requirement of attending research seminars sponsored by our program result in positive impacts (21%–61%), especially in the categories of scientific/engineering efficacy (61%) and scientific/engineering identity (39%). In the past, our department routinely invited speakers outside of our campus to give research seminars in the department. However, few undergraduate students attended these research seminars. Their reluctance to attend may be due to lack of information, conflict of schedules, or perhaps a perceived lack of confidence in understanding the seminar because of the depth of the advanced research topics. Over the past 10 years, we redesigned the seminar series in our department and included entry-level topics on bio-engineering. We have changed the typical practice from inviting speakers only from academics to also inviting speakers from government research labs, researchers in industries, and alumni of our department. The bio-engineering seminar series gave our scholars a broader understanding of potential careers in various aspects of engineering and enabled them to identify potential contacts. The two research-related activities (lab visits and research seminars) are cost-effective to improve their scientific/engineering efficacy and identity. Most importantly, they are activities not limited to a small-scale scholarship program and can be easily adopted to the departmental level.

Other research-related activities included offering REU opportunities and featuring research results on our website. Although only junior and senior students had REU opportunities, the high percentage indicated in the survey (44%) in the category of scientific/engineering efficacy suggests that it is an effective activity (Fig. 4). Less than 14% of the scholars considered attending conferences or featuring their research on a website as having a positive impact. The lower percentages in those two activities may also reflect that not all the scholars had the opportunity to attend a research conference or engage in research and thus not have research results featured on website. The scholars in our program are in all class standings, with approximately half of them are seniors and juniors. If one assumes that only the seniors and juniors in the scholarship program had the REU opportunity, the percentages in those two activities using the scholars who had REUs may be twice of that shown in Fig. 4.

The five community building activities have contributed to help students integrate into the program and campus. Illustrated in Fig. 5, it is evident that having lunch with faculty and other students has the most impact (28%–58%). This may reflect the similar attitudes of students toward mentoring. Although in theory, ME students should be aware of student membership availability in professional societies (American Society of Mechanical Engineers, Society of Woman Engineers, National Society of Black Engineers, Society of Automotive Engineers, Society for the Advancement of Manufacturing and Processing Engineering, etc.), it is unclear to us how many ME undergraduate students applied to or are members of those professional societies. Nevertheless, based on the surveys, joining a professional society is considered as a very important activity to connect with other students. We may conclude that those community building events sponsored by our program effectively improved their sense of integration and belonging to the program and campus.

Eleven of the 36 scholars stated that they would graduate at the end of that semester, and the remaining 25 would continue in the ME S-STEM program. Figure 6 shows the activities they consider as having contributed positively to their retention in the ME program. It is not a surprise again to see that mentoring activities had the most impact (43%–67%) on student's retention, followed by research activities such as lab visits and seminars (52%), and community building activities (24%–43%). On the other hand, the activities in academic help are perceived as less important to students' retention in the program than activities in the other three activity groups. The high retention rate shown in this study is consistent to the data reported previously [36]. Since 2009 when the scholarship program started, 98 of the 110 scholars were retained within the scholarship program and the rest 12 left our scholarship program due to low GPAs, transferring to other institution, or health issues. In the past 12 years, the overall retention rate within the scholarship program is 89.1%. The retention rate is much higher that that in the greater ME population with the 3rd year retention rate approximately 70% [36].

Among the 11 scholars who would graduate from the ME undergraduate program after that semester, 45% of them stated that they would pursue a graduate degree and 55% would work in an industry. Although our sample size is small, the percentage of students intending to pursue a graduate degree after their graduation from our undergraduate program is much higher than the national percentage. Only 10% of mechanical engineering BS graduates nationally pursue graduate MS or Ph.D. degrees [36]. In our department, the overall percentage of students enrolling in a graduate program after receiving their BS degree is typically low (∼10%–15%) due to the high demands for engineering graduates from local/national industry. Shown in Fig. 7, regular faculty/staff mentoring and peer mentoring (36%–55%) and internships (65%) were perceived by the scholars as contributing positively to the graduation decision, followed by community building activities such as lunch with faculty (28%), and research related activities (27%–38%). One of our program outcomes is to encourage our scholars to pursue graduate degree. Currently, more than 70% of the mechanical engineering MS and Ph.D. degree were awarded to international students who received their undergraduate education outside of the USA [36], suggesting dependence on foreign students in U.S. engineering graduate programs. We believe that getting a graduate degree in ME helps students build on the expertise already gained through undergraduate studies and provides them a chance to delve deeper into a particular area of interest. There is a need in STEM industry to have a workforce with advanced degrees. The high percentage of our scholar pursuing graduate degree addresses a national need of talented workforce and shows a positive impact of our program on society.

In this study, we present data that reveal the attitudes and perceptions of the current ME S-STEM scholars based on 36 surveys collected recently. Data analyses showed that the scholars were satisfied with understanding engineering identity, expectations, and sense of belonging. However, further improvements were needed to help students in achieving scientific efficacy and academic integration into the program. The implemented S-STEM activities such as mentoring, research, and internship were viewed favorably by the scholars in helping them to establish their scientific efficacy and engineering identity and to understand their expectations and goals. Community building activities were considered helpful and contributed to integrating into campus life and improving their sense of belonging to the campus and program. It is important to identify cost-effective activities targeting specific deficiency to further improve students' retention and graduation rates in engineering programs.

Parts of the results were presented in the ASEE Annual Conference & Exposition virtual meeting in June 2020.

• Division of Undergraduate Education, NSF S-STEM (Grant No. DUE–1742170; Funder ID: 10.13039/100000172).