The TSTC Harlingen President, Dr. J. Gilbert Leal [Ref 1], leads the institutional effectiveness process, while the Director of Institutional Effectiveness and Research coordinates and directs activities that comply with Policy 1.86, "Institutional Effectiveness and Research," [Ref 2] while the Vice-President's/CFO ensure that their respective divisions are participating in the entire integrated process of planning, budgeting, assessment and evaluation, and use of results that are the harbinger of institutional improvement. All unit supervisors assume responsibility for quality and effectiveness of the program or service, and thus for appropriate evaluation and assessment of their impact upon student development and for their operations. Other employees and students participate in the institutional effectiveness (IE) process through their participation on either college standing committees [Ref 3] or charged with oversight of the effectiveness of various college departments, services, and activities, or, for students, Student Congress[Ref 4]. Additionally, employees, students, and other constituents participate in assessment and evaluation activities by providing feedback to decision-makers for informed decision-making.
The Institutional Effectiveness (IE) Committees
The IE process at TSTC Harlingen is advised by three distinct committees charged with oversight of specific functions of the IE process. Policy 1.86, "Institutional Effectiveness and Research" [Ref 2] describes the functions and charges of the following advisory committees to the IE process:
- Strategic Planning Committee
- Measures and Standards Committee
- Institutional Research Committee
As stated above and outlined in the Policy 1.86, standing committees of the college also play a key role in providing a separate "checks and balances" system to ensure that the operations, efficiency, effectiveness and continuous improvement principles are applied throughout key college programs, services, and activities. The information available in [Ref 5] outlines the various charges of TSTC Harlingen's standing committees. As described in the referenced policy, the IE committees submit recommendations regarding planning, outcomes and performance standard, and institutional research issues to the President's Council. Standing committees submit recommendations pertaining to the areas they are charged with monitoring to the Vice President/CFO responsible for the operations the committee oversees.
Institutional Role and Purpose
As demonstrated in the narrative for Core Requirement 2.4 and Comprehensive Standard 3.1.1, TSTC Harlingen is guided by two statements of purpose: the statutory Purpose Statement [Ref 6] that guides the TSTC System, and the Expanded Statement of Purpose [Ref 7] that is specific to TSTC Harlingen.
The IE process begins with these statements which provide the focus and scope of the college's activities. TSTC Harlingen relies on seven established Indicators of Success (IS) [Ref 8], as the critical success factors that must be accomplished for the college to demonstrate achievement of its mission. Supporting each IS are specific performance measures and standards that support and further define each indicator. The IS, though reviewed and modified periodically, are the base that supports the stability and continuity of the IE process. They set the expectations and standards that are required by external agencies, yet they also set local institutional levels of excellence the college expects of itself. The performance measures are composed of benchmark data, trend analysis, expected outcomes, improvement from baseline data, and simple institutional desire as the standards supporting each IS. Attainment of each performance standard is what TSTC expects to demonstrate accomplishment of its purpose. However, when actual outcomes fall short of the expected performance standards, the IE Measures and Standards Committee ensures that the administration takes appropriate steps to improve performance for all areas of concern.
A biennial Institutional Effectiveness Report Card is published during the spring of even-numbered years to communicate institutional outcomes for all IS for the previous biennium. The most recent edition of the IE Report Card is available through [Ref 9], and in hard copy in the materials sent to each Off-site Review Team member. The publication of this report is one method TSTC Harlingen utilizes to demonstrate accountability to the students, employees, and general public.
Furthermore, to guarantee continued relevance of the statutory, and moreover, the Expanded Statement of Purpose, the Strategic Planning Committee is charged with initiating a comprehensive review, and if necessary, revision to the existing statement, as outlined in Policy 1.77, "Institutional Purpose and Expanded Statement of Purpose [Ref 10]. The cycle for review of the Purpose Statement and Expanded Statement of Purpose is on a five-year timeline. As outlined in Policy 1.77, a study about the perception of the community and the business and industry the college serves is included in the review process. The most recent results of the Public Survey on Institutional Purpose may be reviewed in [Ref 11].
Strategic Planning
Using the metaphor of that TSTC Harlingen's strategic planning process is like a compass, strategic plans guide and articulate the visions and expectations regarding the general direction in which the college hopes to arrive. By uniting the expected role of TSTC Harlingen through its Purpose Statement(s) and TSTC's vision of how it should develop while maintaining excellence and effectiveness, the TSTC Harlingen Strategic Plan provides the general direction the institution will follow in the immediate future.
As prescribed in Policy 1.86, "Institutional Effectiveness and Research," the strategic plan is either completely revised, or updated, every even-numbered year prior to a legislative session where funding for higher education is allocated for a biennial period. In past years, the Legislative Budget Board (LBB) [Ref 12], required all state agencies to submit an "Agency Strategic Plan" during this time period. The document located in [Ref 13] is the last TSTC System Agency Strategic Plan submitted to the LBB and is intended to guide the TSTC System through FY 2005. In 2000, the LBB eliminated its mandate, no longer requiring state agencies to submit strategic plans. Historically, TSTC Harlingen has extracted all relevant visions, expected outcomes, measures, and mandates applicable to the Harlingen campus from the TSTC Agency Strategic Plan to create the TSTC Harlingen Strategic Plan, modeled after the TSTC System blueprint. Now that the TSTC System will no longer be developing a System-wide strategic plan, TSTC Harlingen has adopted a process whereby a TSTC Harlingen-specific Strategic Plan that spans two biennia (four years) will be developed and/or updated every even-numbered year.
Strategic planning integrates institution-wide, research-based planning with a variety of other institutional priorities to include:
- External mandates (including THECB requirements, sponsored program and funded grant goals and objectives, SACS Principles, etc);
- The internal vision of leaders and constituents of the destination they hope to arrive at by the end of the timeframe covered in the Strategic Plan;
- Environmental changes and technical advances that impact college operations;
- Weaknesses identified as "unmet outcomes" in the TSTC IE Report Card; and,
- Goals and objectives of ancillary plans that support and directly feed into the TSTC Harlingen Strategic Plan. These complementary plans, all required for submission to the State of Texas include the Master Plan [Ref 14], the Strategic Enrollment Management Plan [Ref 15], and the Information Technology Strategic Plan [Ref 16].
Taken together, the issues listed above form the linking pins and set the tone and direction of the TSTC Harlingen Strategic Plan. Items included in the Strategic Plan are viewed as priority items and issues that TSTC Harlingen will address during the timeframe specified by the plan.
Operational Planning
Operational Planning takes place at the administrative level after the redevelopment of the TSTC Harlingen Strategic Plan during even-numbered years prior to a legislative biennium. Operational planning is essential to carry the priorities described in the Strategic Plan to the level of implementation, identification of resources, and evaluation of the activity. Operational planning is where budgeting begins in the IE process. The narrative for Core Requirement 2.11 under section "C. Annual Budget," details exactly how the budgeting process for annual and biennial budgets is integrated into the IE process.
Essentially, the Operational Plan is the "roadmap," versus the "compass" in strategic planning, that moves the college directly to the destination it hopes to arrive at within the next two fiscal biennia. The Operational Plan extracts the specific visions, goals, objectives, and activities from the TSTC Harlingen Strategic Plan and includes more local and unit-specific or division-specific needs for large-budget items and any health or safety issues discovered through the cycle of assessment and evaluation. The Operational Plan is designed to provide the details of how TSTC Harlingen will attain the expected outcomes outlined in the TSTC Harlingen Strategic Plan. The Operational Plan is set to be completely redeveloped following the creation of a new TSTC Harlingen Strategic Plan in the Summer 2004. The current TSTC Harlingen Operational Plan can be reviewed in [Ref 16].
As outlined in Policy1.86 "Institutional Effectiveness and Research," the activities that are integrated into the operational planning process include:
- Assigning responsibility for each Operational Plan objective to a President's Council administrator;
- Developing a specific plan of action for each objective within the Operational Plan that assigns personnel, resources, sets measurable outcomes, and determines the units needed to contribute toward the accomplishment of each objective; and,
- Forecasts institutional funding from the Texas legislature for the upcoming biennium and subsequent biennium and sets timelines for when money will be available to fund the priorities set forth in the Operational Plan within the four-year timeline.
Furthermore, it is through the operational planning process that administration determines what units will develop Unit Action Plans (UAPs) that will ensure and support the accomplishment of each objective [Ref 17].
Unit Action Plans (UAPs)
Unit Action Plans are routinely developed and reviewed as an essential facet of the IE planning and budgeting process of the college (See Comprehensive Standard 3.1.1). Its primary purpose is to advance the progress of individual units toward the strategic goals established in the strategic planning process and to advance them in ways determined by the administration in operational planning. Each budgeted unit supervisor presents a report of their outcomes toward their previous plans, summarizes key findings through the assessment and evaluation process, presents budget scenarios based on priority needs and unit visions, and then presents its annual and biennial goals and priorities for the upcoming IE cycle. The UAPs for the current UAP cycle (2003-2005) and the previous IE cycle (2001-2003) are available through [Ref 18]. It is during planning at the unit level that all programs, services, and activities make informed decisions about the priorities for the upcoming biennium based on evaluations and assessment of their operations. Identified weaknesses are expected to be addresses in UAPs as the official mechanism to documents unit improvements and needs. It is here that units demonstrate use of assessment and evaluation results from internal program review (PAIT) [Ref 19], Learning Outcomes Assessment Results [Ref 20], new external mandates, unmet performance standards, and any requirements imposed by program-specific accreditation.
Integrating Budgets and IE
Budgeting is an essential element of both the Operational Plan and the UAPs. Budget presentations for each unit that occur with the presentation of UAPs ensures that administration is aware of funding needed at each level of the institution to ensure that identified priorities from plans, assessment and evaluation results, and new mandates are all given priority when developing the final budgets for the college. The instructions on the template used by units for budget presentations [Ref 21] demonstrates how a unit supervisor presents all identified items that require funding to comply with mandates, operational plan expectation, or significant findings needing immediate attention as a result of internal review processes. By utilizing this process, administrators ensure that all strategic, operational, and unit priorities have funds earmarked for immediate issues and activities, thus providing the foundation for attainment of all plans, while ensuring continuing improvement.
Assessment, Evaluation, and Research
As noted above (and in the narratives for Federal Mandate 4.6; Comprehensive Standard 3.4.1;Comprehensive Standard 3.5.1; and Federal Mandate 4.1), TSTC Harlingen requires ongoing assessment and evaluation of the programs and services of its budgeted units. The program, department, or operational unit responsible for providing a program or service is responsible for the quality of that program or service, and thus for its assessment. However, the Office of Institutional Effectiveness and Research structures the evaluation of several aspects of the educational environment through college-wide research and studies to supplement the assessment of student learning occurring at the unit level.
First and foremost, the Indicators of Success outcomes [Ref 8] form the basis of all research and studies conducted at TSTC Harlingen. The Office of Institutional Effectiveness and Research follows the established IER Planning and Reporting Cycle [Ref 22] outlined for reporting, studies, planning activities, and assessment activities. Assessment and evaluation is generally divided into two separate areas: (1) assessment and evaluation of student learning and development, and (2) surveys and other constituent studies that provide feedback to decision-makers. Both complement each other to provide a clear picture of TSTC's performance.
Assessment and Evaluation
As a result of the need for more accountability in assuring student personal and cognitive growth and development, TSTC Harlingen has developed and is assessing the achievement of student learning and development outcomes in their programs of study, for general education competencies, and for the impact that student development (student activities) programs have on graduates.
The Student Learning Outcomes Assessment Plans (Instructional Assessment Plans) outlined in Comprehensive Standard 3.3.1 and found in [Ref 23]. These plans outline not only how each instructional program supports program outcomes identified in the IS [Ref 8], but the identified skills and competencies that are necessary for success in the workforce. The first report of results for instructional programs with regard to Learning Outcomes Assessment may be reviewed at [Ref 20]. Note the column of each report that discusses the use of assessment findings for improvement in instructional delivery and subsequent student outcomes.
As a complement to the Learning Outcomes Assessment Plans developed by the programs of study, the General Education division has also identified key general education expected competencies, defined them, and developed a General Education Assessment model [Ref 24] to determine the extent to which students attain the desired levels of these general education competency expectations. The narrative for Comprehensive Standard 3.5.1 details the evolution of the identification and subsequent assessment of general education competencies, including the new assessment model for general education schedule to be implemented in Fall 2004 [Ref 25].
The Student Development division (student affairs) also participates in the assessment of student development with regard to the attitudes and behaviors of students that are affected outside the instructional realm of the institution. Through student access of and engagement and participation in student development programs, services, or activities, the division strives to impact the non-academic development of students as a result of their college experience. Student Development Assessment Plans can be reviewed through [Ref 26] and are better explained in the justification narratives for Core Requirement 2.10 and Comprehensive Standard 3.9.3.
Program Review
In addition to learning outcomes assessment, the TSTC Harlingen also conducts extensive program reviews of its instructional and administrative and business operations. Using the Program Assessment and Improvement Team (PAIT) [Ref 27], and following the established cycle of when peer review teams are scheduled to conduct the Exit Reports for each program undergoing program review [Ref 28], TSTC Harlingen ensures that all programs comply with all external and internal mandates and performance standards for their operations that are separate from the assessment of student learning described above. Not only do these reviews identify program weaknesses and strengths, but feed the planning process for units during the next IE cycle. The most recent results of PAIT Exit Reports for programs under review can be viewed at [Ref 19]. The Exit Reports guarantee that a program is supporting the strategic and operational visions of TSTC Harlingen, and that each program is achieving its own unit purpose statement. (Unit Purpose Statements can be viewed on Assessment Plans in [Ref 23] and on UAPs for each unit at [Ref 18].)
For all units that are not in direct contact with students, the Office of Institutional Effectiveness and Research (IER) also conducts periodic internal assessments by users of these programs, services and activities. The primary function of these internal evaluations is to determine impact, user satisfaction, and whether or not these units are effectively accomplishing their stated unit purpose statements. Outcomes to the most recent results to these internal evaluations can be reviewed at [Ref 29]. Again, this information helps build the foundation for future planning and improvement for departments that do not directly impact student learning or development.
Research (Surveys and Other Studies)
The IER Office also conducts a series of surveys, focus groups, and other studies that are either embedded as part of the IE Planning and Reporting Cycle [Ref 22] as well as special studies as requested by the administration, unit supervisors, or a TSTC Harlingen standing committee. The following table details the cyclical studies and surveys of the IE process as well as identifies those special studies conducted over the past few years.
In addition to the institutional studies adopted and administered above, TSTC Harlingen also relies on trend analysis and the data and information that the institution reports to national, state, regional, System, and local entities. As outlined in Policy 1.85, "Data Dissemination" [Ref 30] TSTC Harlingen is required to submit a variety of mandates reports to several external oversight agencies. Additionally, though trend analysis reported in the TSTC Fact Book [Ref 31] and through the On-line data query [Ref 32], all constituents are able to track and evaluate the impact on student enrollment, graduation, persistence, and other key trends to respond accordingly in the planning processes.
Continuing Improvement and Accomplishment of Purpose
The IE cycle, described above, ensures that all units of TSTC Harlingen are engaged in the process of continuous quality and improvement and use of results to determine the direction and guide activities and resource allocation for all levels and facets of the institutions programs, services, and activities. The Institutional Effectiveness Handbook [Ref 33], outlines in detail the processes for all of the functions and participants in the processes described in this narrative. By using research-driven information to form the priorities for subsequent plans, and demonstrating accomplishment of TSTC Harlingen's purpose through achievement of expected outcomes and performance standards contained in the IS [Ref 8], the College not only maintains a process of continual quality improvement, but demonstrates that the institution is effectively accomplishing its mission. This process, in essence, "closes the loop" on issues of identified weakness once performance standards indicate improvement based on effective planning and implementation of goals and objectives designed to correct deficiencies.
A recent study conducted by IER Office analyzed the extent to which college employees were satisfied with the Institutional Research process at the college, and the extent to which they used information provided by the IER Office to plan and improve identified weaknesses. Results can be reviewed in [Ref 34]. Additionally, unit supervisors, administrators, and members of all IE committees participated in a study asking them to rate their satisfaction with the effectiveness of the IE process at TSTC Harlingen. Results show overwhelming satisfaction and understanding of the process, as reported in [Ref 35].
To further demonstrate the extent to which the IE process at TSTC Harlingen has resulted in continuous improvement, a specific site was established [Ref 6] to communicate the most recent impacts of the TSTC Harlingen IE process on all instructional, student development, and administrative operations.