Performance Appraisal Project

Software Engineer Performance Appraisal Data Analysis

Client: SoftApps Corp. (SAC)

Location: Bozeman, Montana, Baltimore Maryland

President and CEO: Ashonnie Starr, President and CEO (Native American)

Key Personnel: Izzy Shorter, COO: Stanley Tall, CIO, Wanda Wynn, VP HR, CFO: Dewey Cheatham,

Description of Client relations to Applied Consulting, Inc.

SoftApps Corp. has been a client of APC since 1985 when we signed a contract for $500,000 to design the primary operations for job classes of software engineers, computer hardware specialists, and programmers for applications software. After several years of expected operating losses, SAC turned a profit in 2000 and established an IPO with the help of Deloitte Financial Consulting. Over the past three years APC has been retained to help with succession planning, management selection, and organizational development. The latest project involves data analysis of the Performance Appraisal data from Software Engineers for the past fiscal year.

Description of client organization:

SoftApps, Corp. (SAC), a multi-state company that develops applications software primarily for business and industry, was incorporated in 1985 by Stan Tall and Izzy Shorter to develop software for PC applications that are user friendly (i.e., consistent with proven, cognitive psychological best practices).

There are two primary locations, Bozeman, MT, corporate headquarters, and Baltimore, MD, a primary sales and distribution facility. There are also regional sales offices in several locations throughout the country. These regional offices are staffed by sales/marketing personnel and a secretarial support person. Total employees: approximately 750, consisting of about 660 software engineers, sales, marketing and distribution, trainers, finance and accounting. Recently, management at SAC has decided to move to a team approach for programmers and software engineers. At the same time, each software engineer project manager will have to directly manage client relations in the development and implementation of the software.

SoftApps has been in a growth phase over the past decade when business began to really take off. Client's users have found that the applications can be learned quickly, and are flexible enough to meet the varying demands of their customer needs. A major attractive feature is the personal support from training and development for on-line help.

The software development and R&D is in Bozeman, while production and distribution is in Baltimore. The company specializes in communications software and workstation special needs.

SAC Mission:

To develop, produce and market applications software products for a wide range of business and industry needs.
To provide products that will be written to accommodate any commonly used platforms and operating systems, and that will be adaptable for use in a variety of organizations.

· To produce high quality software applications for both general and specific (custom made) uses.

To provide on-site client training along with on-line support once users have been trained.
To provide excellence in customer service in development, timely delivery, and continued support for products.

BOZEMAN, Montana: Operations and Production Facilities:

Key Personnel:

Ms. Ashonnie Starr, President and CEO

Mr. Izzy Shorter, Chief Operating Officer (COO)

20 Support personnel: Office mgrs, Secretarial, Maintenance

Mr. Stanley Tall, CIO

4 CIS technicians

Mr. Dewey Cheatham, CFO

6 Accountants and 1 financial advisor
Mr. Willie Gates, Mgr. Product development

      450 Applications Software engineers (approximately- always changing)
         45   Software Engineer Managers
           7 hardware specialists

Ms. Pat McCann, Director of Research and Development

5 Computer science experts

Assignment:

Analyze Performance Appraisal data for software engineers and report findings to SAC management.

Description of situation at SAC and PA data analysis project

HR manager, Wanda Wynn was informed by her astute HR generalist, Cindy File, (who studied I/O psych) that EEOC investigators were inquiring about the possibility that female software developers may be making less than male developers. This prompted the discussion with Carly Simon and Izzy Shorter about whether or not they needed a formal appraisal process to make sure any pay differentials, if they exist, are defensible. Cindy pointed out, of course, that legal issues notwithstanding, its good practice to do the performance appraisals properly especially if they are rewarded on merit and performance.

They all decided it was a smart move to establish a formal performance appraisal system so Izzy decided to borrow an evaluation form from a friend who worked at MicroSoft Corp. However, Cindy pointed out that they should not take an "off the shelf system" but instead developed their own that fits the SAC purposes, starting with a job analysis since job descriptions had not ever been formally done. They all agreed to begin with software engineers (developers). Cindy called on APC for help. The APC consultant agreed to analyze PA data for software engineers for fiscal year 2018.

Not only do the engineers have to demonstrate competency in the technical requirements of the job, each must also work closely with team members and project managers, relating to them effectively and being able to resolve differences in opinions on how to develop the products. Therefore, in addition to capturing the engineer's competency in the technical requirements of the job, the PA must also be designed to assess soft skills.

Prior to this PA Data Analysis project, APC conducted a job analysis for software engineer, the purposes of which were to develop a job description, a recruitment announcement and the Performance Appraisal Rating form PARF that was put in place this past year. This PA was designed to fit into a total organization-wide performance appraisal system to be developed after this project is complete. Thus, comparisons will have to be made across jobs and classes of jobs so that results can be used to set merit pay and used to help make promotional decisions.

Current project: PA Data Analysis for 2019 job of software engineer

APC has been asked to analyze Performance appraisal data from the 2019 for approximately 450 software engineers. The purpose is to compare ratings among three sources of subject ratings: (1) software engineer self- ratings, (2) their managers’ annual rating, (3) a client survey rating, and an (4) objective criterion, project error rate. SAC management would like to determine if there is a correspondence among the PA criteria. This will help to assess the validity for each.

There are two uses for the PA results, one for administrative use and the other, for development purposes. There have been questions raised regarding whether it’s wise to use the same PA for both. SAC will rely on the APC consultant to weigh in on this by providing evidence-based findings regarding the wisdom of doing so.

Management would also like to determine how PA results may differ by demographics such as race and gender. This will help to determine what, if anything will need to be done if there are, in fact, race and/or gender differences. The APC consultant is also expected it weigh in on these issues should differences be found.

Finally, because of the difficulty in recruiting new, highly qualified software engineer applicants, management would like to know if the minimal requirement for a college degree is really needed. Should educational levels not prove to predict higher performance, this would open up a broader pool of potential applicants. Despite the attractiveness of the Bozeman area of Montana (many wish to leave Silicon Valley in CA), the job market remains tight for software engineers.

Mr. Willie Gates, Mgr., Product development, will be working closely with the APC consultant. He has provided the following information on staffing levels and the SAC PA rating Scale. That may be useful for the project.

           450 Applications Software engineers
           45   Engineer Software Managers
            7 hardware specialists

SAC Software Engineer Client Rating Scale* (5 items on a Likert scale (1. Strongly Disagree to Strongly Agree)

Rating Items: “My SAC Software Engineer Associate:”

1. Is a pleasure to work with…

2. Completes projects on time

3. Explains all coding in simple language

4. Responds to all questions in a timely manner

5. Produces high quality programs that meet my needs

*Note: The Client Rating is the mean of the mean of 5 items for 12 customer surveys (one per month randomly selected) for each associate, over the past 12 months before the Performance Review is conducted. (mean scores range is from 1-5erro.)

SAC Software Engineer Performance Appraisal Rating Scale** (5 items on a Likert 5-point scale) Labels:

1= Unsatisfactory; 2= Needs improvement; 3 = Meets expectations; 4 =Exceeds expectations; 5 = Exemplary)

1. Accuracy of code (number of bugs needed to be fixed)

2. Accuracy of design (how many iterations of design is required in general before final approval)

3. Accuracy of quality assurance checks

4. Performs tasks independently

5. Works effectively with SAC colleagues

**Note: The self and manager ratings are each the mean 5 items (range 1-5)

Your assignment is to:

Analyze the PA data and present a written report of findings to SAC management

The SPSS.sav data set you will analyze contains these variables (data definition)

VAR Name	Type	Dec	VAR Label	Value Labels

Gender	Numeric	2	Gender	{1.00, Male}...
Race	Numeric	2	Race	{1.00, Hispanic}...
Tenure	Numeric	2	Years Tenure at SAC	None
Degree	Numeric	2	Education Level	{1.00, High School}...
PA_Self	Numeric	2	PA Self Eval	{1.00, Unsatisfactory}...
PA_Mgr	Numeric	2	PA Eng/Mgr Eval	{1.00, Unsatisfactory}...
Client_rating	Numeric	2	Client PA Rating	{1.00, Strongly Disagree}...
Error_rate	Numeric	2	Percent Error Rate	None

DATA analyses:

1. Demographics: Analyze/Descriptives

a. Frequencies: VARS Gender, Race, Degree

b. Xtabs: Race X Gender

c. Xtabs: Race X Degree

d. Xtabs: Gender X Degree

e. Descriptives: Tenure

2. Results: Analyze/

a. Correlate /bivariate: (with option-Descriptives) PA Self/PA Mgr/Client rating/Error rate/Tenure

b. General Linear model/Multivariate

i. DVs by Gender (option -Descriptives)

c. General Linear model/Multivariate

i. DVs by Race (3 levels) (Post Hoc –move Race -> Post Hoc Tests for: choose Tukey’s -b)

d. General Linear model/Multivariate

i. DVs by Degree (3 levels) (Post Hoc –move Degree -> Post Hoc Tests for: choose Tukey’s -b)

3. Optional:

a. Any stats Geeks are welcome to do a deep dive detective work using MR or any other proc

Guidelines for writing PA Project Report*

Title
Executive summary - Maximum 300 words single spaced summary of the report. Be sure to include the recommendations.
Purpose statement – why the PA is important for SAC.
*Analysis - The analysis must relate to relevant theories and findings regarding the use of performance appraisals in performance management
Findings - describe the results and explain significant findings
Discussion –

Summarize what why/what was done (about two paragraphs)
Explain important findings and discuss implications
Conclusions and recommendations to management (including benefits to SAC)

References - Use APA style for citing references

The PA Report will be a 2000-words or less (not counting tables/figures) single-spaced Project Report, to management.
Submit to the Sakai assignment folder by the due date.

The report must include a one-page executive summary (max 300 words – USE PARAGRAPHS) and a narrative which consists of the following sections: (described in more detail below)

title page (name of project)
Executive summary
Description of the situation that prompted the project (opportunity problem situation)
Introduction

Description of SAC
Purpose of the Report
Relevant literature review to include:

Purpose(s) of PA administrative/developmental
Benefits of valid PA in performance management
Legal issues to be considered

Judgmental biases to avoid
Importance of rater training

Method

Participants (sample size) and crosstabs for demographics (gender/race/degree) and

Descriptive stats for tenure

Describe raters / rating scales used
Description of objective criterion (error rate)
Explanation of aggregation of ratings for client ratings

Results

Descriptives for ratings (and average of three subjective ratings -Self/Mgr/Client)

Correlations among objective and subjective criteria and tenure

Group level analyses

MANOVA/univariate Ratings and objective criterion by gender, race, degree

Discussion

Findings

For each type rater, objective rating

Relationships among rater types and objective criterion

Any group level differences

Potential legal issues that may emerge

Conclusion and recommendations (for how the logistics of the Performance Appraisal should be used)

E.g. how should each of the criteria be used for administrative and/or developmental purposes.

References (APA style)

__________________________________________

Some Issues to consider:

Role of PA in Performance management
Conflicting Dual purposes of PA (administration and development)
Measurement issues: objective v. subjective, ratings in the use of performance management
Legal issues related to the use of PAs
Role of feedback in performance management
Multi-rater issues

7. Explanations for use of Criteria: subjective and objective (reliability/validity)