APPL 645 Personnel Assessment

Data Analysis Project (DAP)

As part of your course requirements, you will complete a Data Analysis Project (DAP). The purpose of the project is to demonstrate your competence in analyzing test validation data and reporting results to management. You will be provided with a data set that includes predictors and criteria for a specific job in an organization.

Rubric for Evaluation

About FASP (client organization)

About POPS (where you work)

Your assignment

Report outline and suggested SPSS analyses

Links to the O*Net job descriptions

Rubric for Evaluation

The DAP Project will account for 20% of your course grade. The Report will be evaluated on the following criteria:

Content

1. Well organized

2. Background literature presented thoroughly and including seminal references

3. Used and understood psychological/psychometric terms

4. Explained instruments and justify how they used (predictors and criteria)

5. Validity and reliability evidence presented (statistics: tables, graphs)

6. The results of the supporting statistical analyses are clearly presented

7. Discussed issues involving adverse impact, applicant reactions, Socially Desirable Responding

8. Described benefits to the organization

9. Used full and proper citation

Client: Farm-A-Suitable Product, Inc. (FASP)

About FASP

FASP was established in 1990 by Wanda Wye and Howdy Doitt, two childhood friends who were very curious young resarchers in bio-geneitcs. One day long ago while playng together in their back yards they became curious about how plants and animals came about. Wanda’s mother told them that both plants and animals grew out of the ground (not wanting to have to explain the details of reproduction to such innocent, young children).Years later, after having completed their bachelor degrees in biology, Wanda and Howdy brainstormed for hours and came up with what they though would be a way to revolutionize food production to feed a planet of 10 billiion[1] hungry people.

They reasoned that planet earth would never be able to provide enough animals for meat to satisfy the insatiatable apetite of 10 million humans. Nor would there be enough farmland on earth to provide agricultural products to sustain such a large world population.

Therefore, by developing innovative genetic manpulation (GM) techniques, they could merge the bio-genetics for animals and plants and in doing so, invent new suitable food products to feed the world’s burdgeoning population.

Therefore, by merging the DNA for plants with animals they would be able to produce both ”agricultual and animal products” that would be deemed ”suitable” (i.e. functionally effective and acceptable) to both the increasing number of vegans and more populus carnivores.[2] Both animal and plant products would be grown in a wide range of climates (from the melting polar caps to the equator) and in types of soil ranging from red clay to the sandy bottoms of the Atlantic and Pacific oceans. Texas ”farms” would grow beef cattle, Nebraska would grow whole grain bread, pancakes, etc.. In the San Jochin valley of California would grow products such as leafy vegetables and citrus fruits. Alaskan fishers would grow salmon in the muddy north Atlantic sea bottom (i.e. ”farm raisied salmon”), and poulty farms wold grow chickens out of of the rich, sandy soil on the eastern shore of Maryland.

To blunt the complaints of vegans opposed to hurting sentient animals, the animals grown out of the ground would have no pain receptors (DNA having been genetically manipulated to remove any sensing neurons). To appease the hunters, farms in the U.S. midweast would still have bighorn sheep, buffalo, deer, etc. that could be stalked and killed (without inflicting pain) by NRA hunters and paleos[3].

The Chicago slaughter houses would no longer be necessary. Beef cattle could be ”chopped up” much like a head of cabbage[4]. The end product would resemble the beef steaks, lamb chops, whatever. Depletion of aquatic life would no longer be necessary given that fish grown out of the sea floor would be plentiful enough (if seeded constantly) to sustain the existing food chain of aquatic life.

Finally, the little children who are brought by their parents on vacations out west to see the cattle roam and the corn swaying in the prarrie fields of Nebraska could still enjoy that wonderful feeling of having ”things like they used to be” when life was good in the U.S.

To bring this all about, Wanda and Howdy conducted some basic and applied researdch in bio-gentices and devised a workable grand theory of life that combined organic and inorganic chemistry. This innovation won them a Nobel prize in 2001 for finding a way to consolidate the genomes of animal and plant life[5].

To accomplish their mission and gear up to produce enough suitable products, FASP management needs to hire more chemists and sales people in the next two years for the company to reach its target of 200 suitable products by 2020. They must bring products to market within that timeframe to get out in front of competitors such as Monsanto Corp and others who are developing state-of-the art genetically modified agricultural seed products.

About POPS

Personnel and Organizatioal Psychology Solutions, LLP (POPS ) has been retained by FASP to provide organizational consulting services. POPS was founded (under a differennt name) in 1889 by the James brothers, William and Henry when Bill was professor of psychology at Harvard. Upon realizaing that neither had the interest nor skills to conduct research, they secured the help of Hugo Monsterberg, an I/O psychologist who turned the company around, making it the most profitable (and only) I/O consulting firm in the U.S. at that time. Most recently, POPS has expanded its services to include talent selection and change management. You, acting in the capacity of a contractual consultant[6], have been assigned to the talent selection project for their new client, FASP.

Hiram Quigley, the HR director at FASP has asked Frank N. Stein, the CEO, to authorizie the hire of 50 more chemists and 30 more sales personnel within the coming year. This hiring is essential for FASP to handle the new business that has resulted from the recent astonishing advances in bio-genetics [7] that have come out of research and development at FASP. Hirim has retained POPS to analyze data from two concurrent criterion-related validity studies, one for chemists and one for sales personnel. The objective is to develop a sound selection procdure that will enable Hiram to bring on board the new chemists and sales respreseantationves.

Unfortunately the consultant[8] who designed the study and collected the data for the two validity studies is no longer under contract. Therefore, POPS has been retained to analyze the data and submit a report of the findings to management.

Your assignment:

Analyze data from a concurrent criterion-related validation study (chemist or sales rep) and report the findings to the FASP senior management team (SMT).

Because HumRowe consultants have already designed the validation study and collected the data you will only need to analyze the data and report the findings to CEO Frank Stein and his senior management team. The tasks to be completed for your assignment are provided below followed by a general outline of the report and suggestions for SPSS data analyses.

As a POPS consultant you will conduct a thorough analysis of the data using SPSS and report the findings in way that they will be understood by FASP managers.

You will be assigned to analyze a current criterion-related validation data set for either Chemist or Sales Representative. The data set will be unique for each junior consultant

About the validation data set:

note: (in Table 1 below)

1. VAR #6 chemex (chemistry exam) is an achievement test, for Chemist (and for Sales Rep, salesexam - is an aptitude test).

2. VARs #14, #15, #16 are Performance Ratings computed by summing 5 Job Duties (1-5 Likert scaled items) to obtain a total Score score with a possible range of 5-25 for each incumbent.

3. VAR #17 for Chemist is Percent of Projects assigned that were completed (average of 12 months), and for Sales Rep, a percentage of sales goals achieved (average of 12 months).

4. Handouts will be provided that show the norms for WPT, WGCTA, NEO -FFI.

5. All other variables definitions are self-evident from the labels

Table 1 Validation Data: for JOB.sav

VAR Num	VAR Name	VAR label	VALUE Labels
1	gender	Gender	{1, Female}...
2	race	Race	{1.00, African Amer}...
3	tenure	Tenure	None
4	cert	Certified	{1.00, Yes}...
5	degree	Degree	{1.00, Bachelor}...
6	chemex	Chem Exam	None
7	wpt	WPT	None
8	wgcta	WGCTA	None
9	neoN	Neo N	None
10	neoE	Neo E	None
11	neoO	Neo O	None
12	neoA	Neo A	None
13	neoC	Neo C	None
14	PAself	PA self rating	None
15	PAsupe	PA super rating	None
16	PApeer	PA peer rating	None
17	pctpro	Pct Projects Completed	None

Specific Details of your assignment::

Describe the validation strategy, and why it was done, i.e. to (1.) improve the quality of hires and (2.) avoid any potential discrimination allegations that could result in litigation from EEOC or other regulatory agencies.
Match the SKAOs from the O*Net job description for whichever job (chemist or sales) you have been asked to work with. Make sure there is a good rationale provided to show that the KSAOs are adequately matched to the predictors. This will require finding empirical research from peer reviewed studies that establish the relationships (construct validity).
Assess the reliability of the criterion for the performance measure (self, supervisor, and peer ratings). Use SPSS Cronbach Alpha or ICC (same resut
Analyze the data to identify if any predictors are biased against protected groups.
Assess the validity of the test battery to include the administered tests and any demographic variables found to be related to the measures.
Recommend an efficient selection procedure that FASP can use for hiring such as temporal sequence for predictors, e.g. tests, interview, etc., and whether to use multiple hurdle or a compensatory strategy for selection.
Be able to give sound reasons (based on I/O principles) for your decisions.
You may be asked to present your findings to the FASP senior management team in a short presentation.
Provide a written report to management that includes an executive summary and any appendices necessary.

Report outline and SPSS analyses:

1. Title: make it definitive as to what was done

2. Executive Summary Guidelines for Writing an Executive Summary

3. Introduction

a. Describe FASP and what they do

b. Describe the need for the validation study

i. Need to hire

ii. Improve selection for the job assigned to you

iii. Prevent legal challenges to the selection procedure

4. Method:

a. Describe the validation strategy and how it was developed: HumRowe:

i. conducted the job analysis (using O*Net!)

ii. Identified the predictors and chose the demographics

iii. obtained self, supervisor and a peer rating of performance (criterion)

iv. obtained an objective criterion

b. Provide a rationale for why predictors were chosen (connect to KSAOs)

c. Describe the sample of incumbents (N and demographics) -see Results

5. Results: present the results, (SPSS output in APA style[9]) including:

Note: Remember not to include nominal variables in the correlation tables!

1. Inspect all data using: Analysis-> Descriptives >Explore

2. Frequencies / crosstabs (categorical) and descriptives (continuous) for all variables

3. Descriptives for all continuous variables

4. Correlations among continuous predictors

5. Compare means and SDs to appropriate norms for WPT, WGCTA, Neo FFI facets

6. Correlations among predicros

7. Correlations among criteria (three PA ratings and Objective criterion

8 Correlations between continuous predictors and criteria (use SPSS ‘with’ statement)

9. Estimate interrater reliability for raters’ criterion judgments

Analyze>Scale>Reliability Analysis>[statistics] ...use ALPHA default

10. Compute mean criterion score for combined raters

Analyze>Transform>Compute variable

11. Calculate ANOVAs (One Way) for grouping variables to:

a. Determine if categorical predictors show mean differences on predictors and criteria (i.e group differences)

b. Determine if race/gender groups differ on means for predictors and/or criteria

c. if so for race, then isolate Blacks and Whites (select only for those two groups)

Data> Select cases> if

12. If adverse impact is evident from ANOVA mean differences, then test for differential prediction ( i,e, significant R² change for interaction term)

then conduct test to determine if slopes differ (moderated MR with gender or two race groups as moderator)

snytax for graphing slopes:

iNSTRUCTIONS FOR GRAPHING Differential Prediction for two levels of race and WPT (assuming you named var for 3 raters (Mean3PA)

to obtain the regression lines for each group

(double left click in graph; in graph insert, click on 6th icon from left (Add Fit Line at Subgroups); close window

13. Correlations between predictors (those to be included in the battery) and total criterion score

14. Conduct multiple R to estimate % variance predicted by battery for each criterion and/or combined criteria (Validation regression analyses (including R² and R² change.)

15. Utility: decide which predictors will be used in final battery

16. Determine how predictors will be used (compensatory/multiple hurdle) and their sequence in selection

7. Discussion

a. Summary paragraph of what was done

b. Description of findings

i. Validity analyses

ii. Any meaningful group differences

iii. Any provisions for addressing adverse impact issues and differential prediction

iv. Recommendations for implementation of selection procedure

1. Sequence of administration

2. Pragmatic considerations

v. Usefulness of findings

1. Improved hiring benefits

2. Protection against legal challenges

vi. Study limitations

c. Conclusion (short recap and recommendations to revalidate in future.

8. References

9. Appendices (as appropriate)

a. Position Description (Appendix A)

b. Selection Protocol (Appendix B)

the data set for the job you will be assigned (either Chemist or Sales Representative) to will be posted in your Sakai assignment folder: DAP Report.

Links to the O*Net job descriptions for the two jobs are:

Chemist 19-2031.00

Sales Representatives, Wholesale and Manufacturing, Technical and Scientific Products 41-4011.00

[1] U.S Department of Agriculture (2009). Estimate of world population by the turn of the next millennium. Paper presented at the 150^th annual Malthus Society convention in New Zealand.

[2] India Office of Overpopulation (2011),. Estimate of the proportion of meat eaters to vegetarians in the new millennium. New Deli conference, Tel Aviv.

[3] The paleo dieters would later come to realize these animals are not real "meat” in the true genetic sense. But then it will be too late.

[4] There would be no blood since all animals would be grown using photosynthesis.

[5] Don’t worry. The human gut would evolve to accommodate the food products, thus making the farm products suitable for human consumption.

[6] All employees at POPS are contractual so the company does not have to provide health benefits.

[7] FASP scientists have devised a means for maniuplation of genes that such that both animals and plants can be grown using photosynthesis and with recycled water.

[8] HumRowe, Inc., a Washington D.C. based consultant was obtained to conduct the study but due to a lack of cooperation from top management at FASP, the company abruptly cancelled the contract.

[9] All tables and figures must be in APA style.