ASSESSMENT REPORT

 

 

DIVISION OF COMPUTER INFORMATION SYSTEMS

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


2004 - 2005 Academic Year


Assessment Evaluation Report

 2004 - 2005

Division of Computer Information Systems

University of Arkansas at Monticello

 

 

 

Evaluation of Unit Assessment Plans

 

1.      Do the academic unit's mission statement and goals flow from and contribute to the University mission statement?

Yes, the division goals contribute to the University’s mission statement.

 

2.      Were the major field assessments based on the mission and goals of the academic unit?

Yes, the major field assessments are based on the Division of CIS’s mission and goals.

 

3.      Are the data being gathered being used for the intended purposes?

Yes, the data we receive aids us in future curriculum revisions and development. 

 

4.      Were the faculty involved in the process?

Yes, due to the small size of our faculty, faculties are involved in every aspect of the assessment, from reviewing, compilation, and modifications of the process.

 

5.      Were multiple measures used in gathering data?

Yes, multiple measures are used from case studies, hands-on programming/networking, written and oral presentations, and team work assignments, in addition to the traditional quantitative testing mechanisms.

 

6.      Were multiple measures used in making programmatic changes?

Yes, all changes from the 2003 CIS Assessment Report have been addressed and included in this annual report.  Further proposed changes have been identified during the 2004 - 2005 assessment period, which will be addressed and reported in the 2005 2006 CIS Assessment Report.

 

7.      Describe feedback to both students and faculty.

Students are made aware in a timely manner throughout the semester of the results of the various tools used through the division’s assessment plan.  Faculty regularly discusses various methods used in assessment and their results.

 


8.      Is the plan cost effective and reflect access, equity, and diversity restrictions in any way?

Yes, the plan is cost effective.

No, the plan does not reflect any restrictions to access, equity, or diversity.

 

9.   How does your unit’s assessment system link to research on learning theories, constructing vs. acquiring knowledge, and active learning strategies?

Students are required to engage in various hands on performance-based learning theories.  Some examples of this are: participation in team-oriented projects, presentations using Microsoft PowerPoint to peers of various cases and programming projects, students building a client-server network environment, and development of web pages.  These active learning strategies link back directly to the division’s assessment plan.

 

10. How are students required to present verbal and written explanations of how their work products demonstrate attainment of publicly stated goals and objectives for their learning?

      Students are required to submit assignments typed in a professional format.  Work is submitted using various methods of communications such as letters, memos, e-mails, presentations, and reports.  Students are required to use Microsoft Project and Microsoft Visio for written and oral presentations of their work.

 

11. How do course syllabi state measurable objectives for student learning and provide for the assessment of students’ academic achievement?

The CIS division is committed to stating course objectives in such a way that they reflect measurable behaviors whose measurement will be included as part of our assessment.

 

12. How are students involved in the assessment process?  Do students participate in your unit’s assessment committee activities?

                     At present time, we do not include students in the committee meetings                                                      

 

 

class=Section2>

CIS MAJOR ASSESSMENT:

 

 

1.      What we planned to do:

 

Student performance in computing is measured by the faculty using a variety of methods including laboratory assignments, hands-on and written quizzes and examinations, student presentations, case analysis and student exit interviews.  We plan to assess students in the following areas: 

 

·    Practical knowledge of various productivity software packages

 

·     Practical knowledge of various programming languages

 

·    Knowledge of information systems development methods and techniques

 

·    Knowledge of data communications and local area networks

 

·     Teamwork problem-solving skills

 

 

We assess the General Education program in its preparation of students for study in the Computer Information Systems major.

 

Assessment procedures are used in the evaluation of General Education elective courses taught by the Division and Continuing Education offered by the Division Computer Information Systems.

 

 

2.  What we did:

 

To assess knowledge of software packages, microcomputer-based productivity software packages during the 2004-2005 academic years include: Windows XP Pro Operating System, Microsoft Internet Explorer 6.0, Microsoft WORD 2002, Microsoft EXCEL 2002 and Microsoft Power Point 2002.  Although technical aspects of each package comprise the core of the learning experience, students are also exposed to myriad managerial concerns and resultant impact on the business environment.  A large number of hands-on assignments are given as homework and as in-class tasks.  Individual instructor prerogative dictates the frequency and type of exams/quizzes.

 

 

To assess knowledge of various programming languages, students are given various assignments that seek to demonstrate any number of standard programming concepts. Each assignment should gradually build upon previous work.  After the problem has been clearly defined, students are to develop logical solutions via tools such as flowcharts, pseudocode, IPO charts (INPUT/PROCESS/OUTPUT), or structure charts utilizing the software package Microsoft® Visio.  The next step in the Program Development Cycle involves running the program with test data and subsequently ridding it of all errors.  Various debugging techniques are employed to ensure clean compile, linkage, and execution phases.  Although documentation is an on-going process, students should understand its importance and ensure that all coded modules are easily maintainable.

 

Ensuring that students have sufficient background in the construction of data files must also be emphasized in beginning programming classes.   This type of exercise reinforces the differences in batch and interactive processing, and allows the student to better understand the concept of job streams and the job control language required in different environments.

 

 

To assess knowledge of information systems development methods and techniques, homework is one of the major assessment tools for this objective.  This work is designed to reinforce textbook concepts and strengthen written communication skills.  All assignments are typed and submitted in a professional format.  Work is submitted using various methods of communications such as letters, memos, reports, or e-mails to ensure students employ the proper format.  Students also use software packages such as Microsoft® Project and Microsoft® Visio to gain experience working with tools used in business to develop information systems.

 

Teaching a class on a modeling type used in information system construction is another assessment measure.  Modeling concepts are often difficult for the student to grasp.  While students have the text and the instructor lectures, many students need more instruction to successfully master a particular modeling technique.  Students are grouped by different modeling techniques and teach a class on that technique.  The group must research various aspects of the technique, prepare hand-outs, and construct exercises designed to reinforce these modeling skills.

 

A case study has been developed which provides students with the opportunity to employ the varied information systems development methods and techniques discussed in class.  This case study assessment tool shows if the students understand the basics of information systems analysis and design.  Students are placed in small groups with each student responsible for a certain section.  They are then graded on their ability to perform several types of analysis, create an analysis manual, and present their case analysis to the class.  Continuing the same case study, students perform several types of design, create a design manual, and present their information system design to the class.  For both of these sections, students perform evaluations of their peers’ work.

 

Traditional objective exams are also given to periodically to assess chapter concepts.

 

To assess knowledge of skills of data communication and networking, there are two degrees of assessment in this class.  Most of the course grade comes from three exams given throughout the semester.  Each exam consists of both objective and subjective questions and is designed to test knowledge, as well as application of the concepts in short-answer and essay-based questions.  Subjective questions must be answered in complete sentences, using an organized and straightforward manner.  This method of testing strengthens critical and logical thinking, and written communication skills.                                                                                          

 

Class participation is another degree of assessment for the class.  Because a portion of the course is lab work, the participation grade is based on involvement in building and troubleshooting the networks while working in a team environment.  In both lecture and lab, questions and comments related to the course topics are encouraged so that, throughout the semester, the class learns from others’ experiences.  One student’s question often leads to other students thinking and applying the book knowledge and classroom definitions into real-world illustrations.  Excessive tardiness and absences also affects the class participation grade.

 

To assess the students teamwork ability, faculty assigned students to two-person teams. Each team was required to complete five complex programming assignments that tested their skills in logic, output procedures and algorithm development.  Teams were required to construct structure charts to show relationships between the various processes, utilizing the software package Microsoft® Visio. Each team member was responsible for the completion of various sections of the program.  The faculty member observed the work ability of each team.

 

To assess the overall CIS program, Student Exit Interviews/Surveys (see Appendix A), Alumni Surveys (see Appendix B), and Employer Surveys (see Appendix C) were designed in order to produce quantitative data.  Student, alumni, and employer surveys were distributed and responses analyzed. 

 

To assess the General Education Electives, microcomputer-based productivity software packages during 2004 - 2005 academic years include: Windows XP Pro Operating System, Microsoft Internet Explorer 6.0, Microsoft WORD 2002, Microsoft EXCEL 2002, and Microsoft Power Point 2002.  Although technical aspects of each package comprise the core of the learning experience, students are also exposed to myriad managerial concerns and resultant impact on the business environment.  A large number of hands-on assignments are given as homework and as in-class tasks.  Individual instructor prerogative dictates the frequency and type of exams/quizzes.

 

General Education elements were assessed for the Computer Information Systems major, using Senior Project, a capstone course which requires the student to formulate, code, test, document, and present a major computer business model.  The course is divided into four sections: Problem Identification, Analysis & Design, Programming, and Documentation.  For each of these sections, the following general education areas are examined: reason critically, analyze objectively, think creatively, perceive assumptions, make judgments, construct arguments, write/speak effectively, and listen/observe perceptively (see Appendix D).  Each factor is ranked by the faculty member during the various phases of the project.  This subjective ranking is done on a scale of one (1) to five (5) with one being low and five being high.  All scores for all students for the academic year are averaged to produce yearly calculations.

 

To assess Continuing Education offered by the Division, the faculty will use a division evaluation form for assessment purposes (see Appendix E) when the Computer Information Systems division offers continuing education courses.

 

3.  What we learned:

 

In assessing the software package knowledge, reading comprehension deficiencies are becoming more apparent as the complexity of the software increases.  Students have difficulties following the step-by-step procedures outlined in the textbook’s hands-on exercises and assignments for all of the productivity software packages.  Mathematics deficiencies surface most commonly when spreadsheet software is being taught.  Otherwise, students seem to naturally gravitate toward microcomputer-based tasks, and continue to find the work intriguing and satisfying for real-world application.  Students tend to excel in the more creative aspects of the individual applications. 

 

In assessing the students' knowledge of programming languages, Some students become myopic when attempting to solve a new problem and fail to understand the importance of taking a broader view.   Hence, the Program Development Cycle can not be stressed enough.  As in the analysis stage, students need to inspect output needs first then verify what input is available and lastly what processing needs to take place for a given problem.  When executing their programs, students initially see compiler diagnostics as cryptic, but with experience most students begin to interpret the messages quickly; however, they continue to struggle with logical errors.  Students continue to believe that when their programs produce output that this alone indicates perfection.  In this regard, sufficient attention to detail remains difficult for some to grasp.  On the job, this trait will be immediately detected by management, with devastating consequences to the employee.  Likewise, students’ inability to follow step-wise refinement, the process of breaking larger tasks into more manageable sub-tasks surfaces on occasion.  The lack of basic mathematical skills continues to be the greatest barrier to success for beginning students majoring in CIS. 

 

In assessing the students' knowledge of information systems development methods and techniques, having students do evaluations of each other continues to work well.  The same comment coming from several peers rather than just one from the instructor makes more of an impression on the student.  Usually, the improvement from the first presentation to the second presentation is notable.  Having the students do evaluations of each other make them much more aware of things to look for not only in others but also in them.

 

Students have a tendency to be afraid of failure when doing something new.  They think their work should look just like everyone else’s work.  In the CIS industry, “thinking outside the box” is encouraged.  Getting students to realize that there can be more than one “right” answer is paramount.  Giving students the confidence to fail is a major endeavor.

 

In assessing the students' ability in data communications and networking skills, students like the opportunity to discuss exam questions.  By spending time reviewing the exams, the class is encouraged to compare the correct and incorrect answers, thus reinforcing oral communication, reasoning, and logic skills.  This also serves as a chance to tie previous concepts to those that will appear in upcoming chapters and to reinforce concepts before entering into new material.

 

This is a comprehensive course that continues to build upon its concepts.  Information in early chapters is still used near the end of the semester.  These topics are also applied in the computer laboratory setting.  Students have a greater understanding of the course topics after they have applied their knowledge in the computer laboratory.  While working in teams to construct the networks, students share knowledge and experience, giving each team member an opportunity to employ the course concepts.

 

When assessing the students' ability to work in a team situation, overall, faculty did not observe many problems within the team structures. The first program assignment required more lab time, in order for students to gravitate to a team work mentality.  Team members were able to communicate the direction and content of the problems at hand.   Students indicate they enjoy working as a team and learning different points of view on analyzing the various problems.  Having a team member to rely on for coding various parts of the program, then pulling the entire program together has been beneficial.  Faculty found the students pulled together within their teams to combat these issues and completed all assignments in a timely manner.

 

In assessing the overall CIS program, the results of the various instruments for assessing the CIS program are as follows:

·        Student Exit Interviews.   Itemized in the free form questions, the strengths of the CIS program are faculty, teamwork, and curriculum in that order. Most of the comments indicate students want more advanced CIS courses, various topic-based seminar courses, and the need for more computers and lab areas was predominately mentioned.

 

Student Exit Survey.  With twenty-two completed surveys by graduating seniors, the quantitative portion of the student exit survey indicated an ‘excellent’ rating with regards to the CIS Learning Objectives.   Objective 5 - - Team-work Problem Solving was rated ‘outstanding’ by the seniors and was also one of the foremost comments on the free form questionnaire.  Analysis regarding the students responses to three skills development areas, Critical Thinking, Communication, and Written Communication the students felt the preparation in these skills was excellent and had benefited them in their college experience. The overall experience with advising, facilities, and library holdings remain in the excellent range.  The survey in addition addressed satisfaction with major course content and supportive course requirements indicating an ‘excellent’ range for major course work.  With Systems Analysis and Design, and our capstone course Application Software Development Project receiving an ‘outstanding’ rating.  In addition,   the Supportive Requirements received an ‘excellent’ rating,   a significant increase over last year’s survey.

 

See Appendix A for Student survey, raw data, analysis of the Student Survey, and comments from the Student Survey.

 

·        Alumni Survey.    Fifty-nine surveys were mailed to 1999 (16), 2001 (20), and 2003 (23) graduates.  Ten completed surveys were returned; one from the 1999 class, six from the 2001 class, and three from the 2003 class. Six respondents were female and four were male.  Two surveys were returned as undeliverable.    Six respondents are working in the CIS field as programmers, analysts, and database administrators.  Eight respondents indicated that current employment is directly related or somewhat related to their CIS degree.  Nine respondents indicated that the CIS degree prepared them for their position and eight respondents believed that the CIS degree enhanced their prospects for future advancement. 

 

In each instance, the alumni felt that the various Learning Objectives of our CIS program helped prepare them to expand their knowledge in the computing field and the emphasis on the  various areas were ample.

 

With regards to Supportive Requirements, the survey indicated that each requirement helped prepare them in the various topics and that the emphasis was ‘about right’ for each. 

 

See Appendix B for Alumni survey, raw data, analysis of the Alumni Survey, and comments from the Alumni Survey.

 

 

·        Employer Survey.  The model curriculum identifies 39 items in five categories that measure the employer’s perceived importance of these information systems employee skills.  The respondents were asked to indicate the level of importance of each skill item for them and to rate the level of competence of UAM graduates on each skill item.  Skill items related to each of the learning objectives and each of the supportive requirements were aggregated to develop a composite test item for each assessment category.

 

One-hundred-twenty surveys were mailed to businesses in the southeast region of Arkansas where the university recruits students.  Nineteen completed surveys were returned representing eight different types of businesses.  Thirteen surveys were returned as undeliverable.

 

When evaluating the employer response with reference to the various Learning Objectives of our CIS program, the employers indicated the UAM graduate competence level exceeded the employers rating of the importance of the Objective for Programming Languages, IS Development, and Local Area Networking.  Concerning Productivity Software Packages and Teamwork Problem-Solving Skills the employer’s expectation of the objective and the UAM graduates competency level were equivalent.

 

With regards to Supportive Requirements, the survey indicated the UAM graduate competence level was not significantly different from the employers rating of the importance of each of the requirements.

 

See Appendix C for Employer survey, raw data, analysis of the Employer Survey, and comments from the Employer Survey.

 

 

In assessing the General Education Electives, mathematics deficiencies surface most commonly when spreadsheet software is being taught.  Otherwise, students seem to naturally gravitate toward microcomputer-based tasks, and continue to find the work intriguing and satisfying for real-world application.  Students tend to excel in the more creative aspects of the individual applications.

 

When assessing the General Education elements for the Computer Information Systems major, (see Appendix D), overall scores ranged from 3.28 (Listen/Observe Perceptively) to a high of 3.42 (Speaking Effectively), placing all general education skills in the Moderate level of preparedness. After departmental observations and anecdotal evidence from employers, an additional oral communication class and a writing class were placed in the Supportive Requirements section of the CIS program.   In comparing previous data, our present findings indicate an increase in all aspects of the general education skills tested.  However, while the 2004 scores are all higher than the 2003, it is too soon to tell if these numbers are significant. 

 

The assessment of Continuing Education offered by the Division was not done during the 2004 - 2005 academic year, due to non-offerings of courses.

 

4.  What we recommend we do and/or what action have taken and/or plan to take:

 

During the assessment process the following proposed changes were identified: 

 

·        The employer survey was a general mailing to area businesses. We will do a more focused survey for next year that will target businesses that we know have hired our CIS graduates.

 

·        Computer laboratory work continues to be expanded, to encompass technology changes and student interest.  The number of computers available will also need to expand in order to accommodate smaller student groups and an increase in hands-on opportunities as mentioned in the Student Exit Survey.        

 

·        Faculties are reviewing textbooks for the Micro Applications course.  Considerations of the text will be based on Micro Applications being a component of the General Education curriculum.  Text should be hands-on based instead of theory-based, which has been the text of choice in the past.  Faculties anticipate alleviation of some of the problems we have observed by students in this course. The faculties are also investigating the use of the Electronic Testing Service (ETS) new Information and Communication Technology (ICT) Literacy Assessment instrument to provide an objective “outside” validation of learning outcomes.

 

·        As courses are moved from CIS 4723 Seminar status to assigned CIS 3XX3/4XX3 courses, the Objectives and Minimum Content will be developed for these classes and will be on file in the CIS Office.

 

·        Seminar courses will continue to be developed to respond to observations in the Student Exit Survey.

 

·        Additional funding/increase in budget to employ at least two student workers to assist in the various capacities within the CIS Division. 

 

Computer Information Systems (CIS) Major

 

Mission:        

 

The mission of the Division of Computer Information Systems is to prepare students to assume dynamic roles as analysts and designers who will provide the professional insight required for building the information systems of the future.

 

 

Goals:

 

The Bachelor of Science Degree in Computer Information Systems is designed to provide students with an understanding of application software and language programming, to prepare students to write and manage database programs and networks, and to prepare students for advanced study.

 

                                                                       

Learning Objectives:

 

The student who graduates with a Bachelor of Science Degree in Computer Information Systems should meet the following learning objectives.

 

·        Practical knowledge of various productivity software packages

 

·        Practical knowledge of various programming languages

 

·        Knowledge of information systems development methods and techniques

 

·        Knowledge of data communications and local area networks

 

·        Teamwork problem-solving skills

 

 

 


Practices Used to Assess Conceptual Elements of the Computer Information Systems Major:

 

Student performance in computing is measured by the faculty using a variety of methods including:  laboratory assignments, hands-on and written quizzes and examinations, and student presentations.  The academic unit continuously monitors, evaluates, and refines each course in the curriculum to ensure that graduates emerge from not only a quality CIS program, but from one which is regionally competitive.

 

Practices Used to Assess Computer Information Systems Program:

 

Student exit interviews/surveys, alumni surveys, and employer surveys are used to assess the Computer Information Systems program.

 

Student Surveys:

Each student will complete an exit interview/survey during the final semester of instruction.  The instrument will address the student's computer information systems knowledge set, satisfaction with their program course of study, satisfaction with the administration of the program, and perceptions of program areas of excellence and areas of deficiency.  See Appendix A.

 

Alumni Surveys:

The instrument will address the alumni’s satisfaction with their program course of study, satisfaction with the administration of the program, and perceptions of program areas of excellence and areas of deficiency.  See Appendix B.

 

Employer Surveys:

The instrument will address the employer’s satisfaction with the knowledge set of our graduates, program course of study, and perceptions of program areas of excellence and areas of deficiency.   See Appendix C.

 

Career Outcomes:

The following criteria will be used to assess how successful our Computer Information Systems graduates are doing outside the university:

 

Has the student obtained employment in a timely manner?

 

Has the student been admitted to a graduate program?

 

This information may be obtained via word-of-mouth, the news media, and follow-up surveys of graduates.  The Computer Information Systems faculty will share individually collected information with the rest of the faculty on the career outcomes of Computer Information Systems majors.

 

 

Practices Used to Assess General Education Elements of the Computer Information Systems Major:

    

The capstone course, CIS 4633 Application Software Development Project will be used to assess how well the General Education program prepared students for study in the Computer Information Systems major.  The factors that will be used to assess the General Education program are critical thinking, research skills, oral communication, and writing capabilities.  Each factor is ranked, by the faculty member, during the various phases of the senior project.   The assessment is used to reflect the performance of the student on the various factors.  See Appendix D

 

 

Practices Used to Assess General Education Elective courses taught by the Division of Computer Information Systems:

 

The Computer Information Systems faculty will use the assessment procedure for the learning objective “practical knowledge of various productivity software packages” to evaluate the general education mathematics/science/technology elective courses offered by the division.

     

 

Practices Used to Assess Continuing Education offered by the Division of Computer Information Systems:

 

The faculty will use a division evaluation form for assessment purposes when the Computer Information Systems division offers continuing education courses.   See Appendix E.

 



Summary of Learning Objectives

 

CIS Assessment Analysis

2004 - 2005

 

Learning Objective:

 

            Practical knowledge of various productivity software packages

 

Description of assessment procedure:

 

Microcomputer-based productivity software packages during the 2004-2005 academic years include: Microsoft Windows XP Pro Operating System, Microsoft Internet Explorer 6.0, Microsoft WORD 2002, Microsoft EXCEL 2002 and Microsoft Power Point 2002.  Although technical aspects of each package comprise the core of the learning experience, students are also exposed to myriad managerial concerns and resultant impact on the business environment.  A large number of hands-on assignments are given as homework and as in-class tasks.  Individual instructor prerogative dictates the frequency and type of exams/quizzes.

 

Course(s) in which conducted:

 

            CIS 2223 Microcomputer Applications

 

Description of findings:

 

Reading comprehension deficiencies are becoming more apparent as the complexity of the software increases.  Students have difficulties following the step-by-step procedures outlined in the textbook’s hands-on exercises and assignments for all of the productivity software packages.  Mathematics deficiencies surface most commonly when spreadsheet software is being taught.  Otherwise, students seem to naturally gravitate toward microcomputer-based tasks, and continue to find the work intriguing and satisfying for real-world application.  Students tend to excel in the more creative aspects of the individual applications.

 

Feedback procedure(s):

 

All assignments were corrected and returned to the students in a timely manner.  Corrections were examined and explained.  Students had multiple opportunities to demonstrate their knowledge of each package through hands-on assignments and exams so that a relatively solid understanding of each was obtained.

 

Proposed changes, if any:

 

We will update the teaching tool to Microsoft Office Suite 2003 for fall 2005 Semester.  The faculty is also investigating the use of the Electronic Testing Service (ETS) new Information and Communication Technology (ICT) Literacy Assessment instrument to provide an objective “outside” validation of learning outcomes.


Summary of Learning Objectives

 

CIS Assessment Analysis

2004 - 2005

 

Learning Objective:

 

Practical knowledge of various programming languages

 

Description of assessment procedure:

 

Students are given various assignments that seek to demonstrate any number of standard programming concepts. Each assignment should gradually build upon previous work.  After the problem has been clearly defined, students are to develop logical solutions via tools such as flowcharts, pseudocode, IPO charts (INPUT/PROCESS/OUTPUT), or structure charts utilizing the software package Microsoft® Visio.  From the logical design of a problem, students write programming code using the syntax of an appropriate language.  The next step in the Program Development Cycle involves running the program with test data and subsequently ridding it of all errors.  Various debugging techniques are employed to ensure clean compile, linkage, and execution phases.  Although documentation is an on-going process, students finish their projects by ensuring that all necessary documentation is completed. For future maintenance, this includes requisite internal comments and any required external charts as noted above.

 

            Ensuring that students have sufficient background in the construction of data files must also be emphasized in beginning programming classes.   This type of exercise reinforces the differences in batch and interactive processing, and allows the student to better understand the concept of job streams and the job control language required in different environments.

 

Course(s) in which conducted:

 

            CIS 3443 Object-Oriented Programming Languages

            CIS 3423 COBOL

           

Description of findings:

 

Some students become myopic when attempting to solve a new problem and fail to understand the importance of taking a broader view.   Hence, the Program Development Cycle can not be stressed enough.  While in the analysis stage, students need to carefully consider the required output before becoming too involved in particular processing techniques. Likewise, students often do not take enough time inspecting the input and noting its importance in the input/process/output/storage cycle.  When executing their programs, students initially see compiler diagnostics as extremely cryptic, but with experience most students begin to interpret the messages quickly; however, they continue to struggle with logical errors.  Students continue to believe that when their programs produce output that this alone indicates perfection.  In this regard, sufficient attention to detail remains difficult for some to grasp.  On the job, this trait will be immediately detected by management, with devastating consequences to the employee.  Likewise, students’ inability to follow step-wise refinement, the process of breaking larger tasks into more manageable sub-tasks surfaces on occasion.  The lack of basic mathematical skills continues to be the greatest barrier to success for beginning students majoring in CIS. 

 

Feedback procedure(s):

 

All assignments were corrected and returned to the students in a timely manner.  Corrections were reviewed in class and on occasion students were asked to rework an assignment or to modify it according to some criterion.  When necessary, concepts were re-taught. In the object oriented class, the students were often asked to demonstrate their programs with specific data.  This helps reinforce the assignment and allows the students to test data situations that they may not have considered. Much time is spent with students outside the programming class environment.  Besides helping with logical problems at office hours, students often require assistance from instructors in a computer lab environment.  CIS majors have always been most appreciative for this kind of individualized support and realize how fortunate UAM students are in this regard.  In addition, these classes often have the opportunity to develop appropriate data files for source code which has proven a very worthy exercise.  Emphasis should continue to be placed on this aspect of the job stream.

 

Proposed changes, if any:

           

With recent changes in prerequisites for these courses, there are no additional changes at this time.


Summary of Learning Objectives

 

CIS Assessment Analysis

2004 - 2005

 

Learning Objective:

 

            Knowledge of information systems development methods and techniques

 

Description of assessment procedure:

 

Homework is one of the major assessment tools for this objective.  This work is designed to reinforce textbook concepts and strengthen written communication skills.  All assignments are typed and submitted in a professional format.  Various methods of communications such as letters, memos, reports, or e-mails are employed to ensure students employ proper business formats.  Students also use software packages such as Microsoft® Project and Microsoft® Visio to gain experience working with tools used in information systems develop.

 

Since modeling concepts are often difficult to grasp, students are grouped by different modeling techniques and teach a class on that technique.  The group must research various aspects of the technique, prepare hand-outs, and construct exercises designed to strengthen these modeling skills.  Developing and teaching a class over a modeling method used in information system construction is another assessment measure.

 

A case study provides students with the opportunity to employ varied information systems development methods and techniques.  This case study assessment tool uses a manuals and presentations to show if students understand the basics of information systems analysis and design.  To further enhance the learning process, students not only examine a system from the Information Technology (IT) side but also from the user standpoint.  Students are placed in small groups who become IT for one group and the User for another group.  As IT, the groups must perform the analysis and design, produce documentation manuals, and conduct presentations for their users.  As Users, each group must construct their mission (in essence, create the case), answer questionnaires, and evaluate manuals.

 

Traditional objective exams are also given periodically to assess concepts.

 

Course(s) in which conducted:

 

            CIS 3523 Systems Analysis & Design

 


Description of findings:

 

Modeling techniques continue to be a difficult concept to grasp for many students.  Having the students teach a concept provide many research and application skills.  However, many students just go to the Internet, pull off something, and proceed to read it to the class during their presentation.  To eliminate this boredom in the future, students will have to develop a scenario then work out the answer for use as an exercise when teaching.

 

Students evaluating each other continue to work well.  The same comment coming from several peers rather than just one from the instructor makes more of an impression on the student.  Usually, the improvement from the first presentation to the second presentation is notable.  This evaluation process also makes a student much more aware of things to look for not only in others but also in themselves.

 

Students have a tendency to be afraid of failure and also to think their work should look just like everyone else’s work.  Since independent thought in the CIS industry is encouraged, getting students to realize that there can be more than one “right” answer is vital. 

 

Feedback procedure(s):

 

All homework assignments are graded for not only content but also for grammar, syntax, and presentation.  Comments for improvement and/or encouragement are in abundance on individual papers.  Items missed by multiple students are then addressed during class.

 

Group member evaluations and instructor observation are combined to provide the student with feedback for his/her teaching of a modeling technique.  Students receive individual comment sheets addressing their presentation skills, content knowledge, and communication skills.

 

Both the manual and presentation are scored for the case study.  The manual is awarded points based on grammar and spelling, the overall appearance, the consistency of the sections, the content of the data, and the order of the manual.  Both the User group and the instructor provide written evaluations of the manual.  As for the presentation, points are earned based on the consistency of the slide show presentation, the length of the presentation, the information presented, and the ability of the group to clearly present their information.  Students provide written evaluations consisting of at least one thing the presenters did well during the presentation and one thing they could have done better. These evaluations are collected, collated and presented to the individual.  Students are expected to use these evaluations to construct an improvement plan for future presentations.

 

Traditional examinations are scored and returned promptly.  Incorrect items are noted and clarified at that time.  The instructor examines frequently missed questions and may re-teach or modify the teaching of that subject.

 

Proposed changes, if any:

 

None at this time.


 

Summary of Learning Objectives

 

CIS Assessment Analysis

2004 - 2005

 

Learning Objective:

 

            Knowledge of data communications and local area networks

 

Description of assessment procedure:

 

There are two degrees of assessment in this class.  Most of the course grade comes from three exams given throughout the semester.  Each exam consists of both objective and subjective questions and is designed to test knowledge, as well as application of the concepts in short-answer and essay-based questions.  Subjective questions must be answered in complete sentences, using an organized and straightforward manner.  This method of testing strengthens critical and logical thinking, and written communication skills.                                                           

 

Class participation is another degree of assessment for the class.  Because a portion of the course is lab work, the participation grade is based on involvement in building and troubleshooting the networks while working in a team environment.  In both lecture and lab, questions and comments related to the course topics are encouraged so that, throughout the semester, the class learns from others’ experiences.  One student’s question often leads to other students thinking and applying the book knowledge and classroom definitions into real-world illustrations.  Excessive tardiness and absences also affects the class participation grade.

 

Course(s) in which conducted:

 

            CIS 4503 Business Data Communications

 

Description of findings:

 

Students like the opportunity to discuss exam questions.  By spending time reviewing the exams, the class is encouraged to compare the correct and incorrect answers, thus reinforcing oral communication, reasoning, and logic skills.  This also serves as a chance to tie previous concepts to those that will appear in upcoming chapters and to reinforce concepts before entering into new material.

 


This is a comprehensive course that continues to build upon its concepts.  Information from early chapters is still used near the end of the semester.  These topics are also applied in the computer laboratory setting.  Students have a greater understanding of the course topics after they have applied their knowledge in the computer laboratory.  While working in teams to construct the networks, students share knowledge and experience, giving each team member an opportunity to employ the course concepts.

 

Feedback procedure(s):

 

Exams are assessed and returned in a timely manner.  Time is spent reviewing the exam, allowing students an opportunity to justify answers on selected questions, thus continuing the learning process; in order for a student to justify his/her answer, there must be some understanding of the topic being discussed.

                                                                                   

Proposed changes, if any:

 

Computer laboratory work continues to be expanded, to encompass technology changes and student interest.  The number of computers available will also expand to accommodate smaller student groups and an increase in hands-on opportunities.   

 

                                                                                                           


                                                                                                           

 

Summary of Learning Objectives

 

CIS Assessment Analysis

2004 - 2005

 

Learning Objective:

 

            Teamwork problem-solving skills

 

Description of assessment procedure:

 

Faculty assigned students to two person teams.  Each team was required to complete five programming assignments that tested their skills in algorithm development, logic, and output procedures.  Each team member was responsible for the completion of various sections of the program.  Teams were required to construct structure charts to indicate relationships between the various processes, utilizing the software package Microsoft® Visio.   The faculty member observed the work ability of each team. 

 

Course(s) in which conducted:                                     

 

            CIS 3553 Advanced COBOL

 

Description of findings:

 

Overall, faculty did not observe many problems within the team structures. The first program assignment required more lab time, in order for students to gravitate to a team work mentality.  Team members were able to communicate the direction and content of the problems at hand.   Students indicate they enjoy working as a team and learning different points of view on analyzing the various problems.  Having a team member to rely on for coding various parts of the program, then pulling the entire program together has been beneficial.  Faculty found the students pulled together within their teams to combat these issues and completed all assignments in a timely manner.

 

Feedback procedure(s):

 

Faculty observed and visited with team members to ensure compatibility.

 

Proposed changes, if any:

 

            None

                                   

Summary of Learning Objectives

for General Education Electives

 

CIS Assessment Analysis

2004 - 2005

 

Learning Objective:

 

            Practical knowledge of various productivity software packages

 

Description of assessment procedure:

 

Microcomputer-based productivity software packages during the 2004 - 2005 academic year include: Microsoft Windows XP Pro Operating System, Microsoft Internet Explorer 6.0, Microsoft WORD 2002, Microsoft EXCEL 2002 and Microsoft Power Point 2002.  Although technical aspects of each package comprise the core of the learning experience, students are also exposed to myriad managerial concerns and resultant impact on the business environment.  A large number of hands-on assignments are given as homework and as in-class tasks.  Individual instructor prerogative dictates the frequency and type of exams/quizzes.

 

Course(s) in which conducted:

 

            CIS 2223 Microcomputer Applications

 

Description of findings:

 

Reading comprehension deficiencies are becoming more apparent as the complexity of the software increases.  Students have difficulties following the step-by-step procedures outlined in the textbook’s hands-on exercises and assignments for all of the productivity software packages.  Mathematics deficiencies surface most commonly when spreadsheet software is being taught.  Otherwise, students seem to naturally gravitate toward microcomputer-based tasks, and continue to find the work intriguing and satisfying for real-world application.  Students tend to excel in the more creative aspects of the individual applications.

 

Feedback procedure(s):

 

All assignments were corrected and returned to the students in a timely manner.  Corrections were examined and explained.  Students had multiple opportunities to demonstrate their knowledge of each package through hands-on assignments and exams so that a relatively solid understanding of each was obtained.

 

Proposed changes, if any:

 

We will update the teaching tool to Microsoft Office Suite 2003 for fall 2005 Semester.  The faculties are also investigating the use of the Electronic Testing Service (ETS) new Information and Communication Technology (ICT) Literacy Assessment instrument to provide an objective “outside” validation of learning outcomes.


Summary of the General Education Elements

of the Computer Information Systems Major

 

CIS Assessment Analysis

2004 - 2005

 

Learning Objective:

 

            Assessment of a CIS student's general education knowledge

 

Description of the assessment procedure:

 

Senior Project is a capstone course which requires the student to formulate, code, test, document, and present a major computer business model.  The course is divided into four sections: Problem Identification, Analysis & Design, Programming, and Documentation.  For each of these sections, the following general education areas are examined: reason critically, analyze objectively, think creatively, perceive assumptions, make judgments, construct arguments, write effectively, speak effectively, and listen/observe perceptively.  Each factor is ranked by the faculty member during the various phases of the project.  This subjective ranking is done on a scale of one (1) to five (5) with one being low and five being high.  All scores for all students for the academic year are averaged to produce yearly calculations.

 

Course(s) in which conducted:

 

            CIS 4633 Application Software Development Project (aka Senior Project)

 


Description of findings:

 

Following is the table showing the results:

 

Reason Critically

Analyze Objectively

Think Creatively

Perceive Assumptions

Make Judgments

Construct Arguments

Write Effectively

Speak Effectively

Listen/Observe Perceptively

Problem Identification

3.63

3.67

3.58

3.63

3.63

3.58

3.71

3.54

3.58

Analysis & Design

3.00

2.98

3.02

2.98

2.98

3.00

3.04

3.02

3.04

Programming

3.30

3.32

3.43

3.29

3.33

3.30

3.37

3.33

3.23

Documentation

3.43

3.40

3.48

3.35

3.39

3.30

3.43

3.80

3.25

Overall Assessment

3.34

3.34

3.38

3.31

3.33

3.30

3.39

3.42

3.28

1=Poor, 2=Moderately Poor, 3=Moderate, 4=Moderately High, 5 = High

 

Description of findings (cont.):

 

The overall scores range from a low of 3.28 (Listen/Observe Perceptively) to a high of 3.42 (Speak Effectively), placing all general education skills in the Moderate level of preparedness.  While the 2004 scores are all higher than the 2003, it is too soon to tell if these numbers are significant. 

 

 

 

Reason Critically

Analyze Objectively

Think Creatively

Perceive Assumptions

Make Judgments

Construct Arguments

Write Effectively

Speak Effectively

Listen/Observe Perceptively

2003 Overall

2.98

3.10

3.21

2.78

2.85

2.89

2.98

N/A

3.0

2004 Overall

3.34

3.34

3.38

3.31

3.33

3.30

3.39

3.42

3.28

 

 

 

Proposed changes, if any:

 

None at this time.