Evidence PA Core Standards Are Failing Students

Below shows how Pennsylvania students did on standardized tests before the implementation of the PA Core Standards versus how the same group of students scored after the implementation of the PA Core Standards.

8th Grade PSSA 2010-2011 Pre-PA Core Standards	Keystone Exam Results 2014 Post-PA Core Standards
Math 126,486 Students 50.2% Advanced 26.7% Proficient 11.7% Basic 11.4% Below Basic	Algebra 1 203,357 Students    14.6% Advanced 25.5 % Proficient 44% Basic 16% Below Basic
Reading 127,125 Students 57.7% Advanced 24.1% Proficient 9.3% Basic 8.9% Below Basic	Literature 143,298 Students 5.9% Advanced 46.5% Proficient 35.1% Basic 12.5% Below Basic
Science 127,075 Students 24.3% Advanced 34.0% Proficient 18.9% Basic 22.9% Below Basic	Biology 164,757 Students 17.7% Advanced 23.7% Proficient 31.7% Basic 26.9% Below Basic

The PSSA's cost taxpayers $58.3 Million dollars this year.   Teachers, parents and students get no educational student-specific information from these tests as scores come in September AFTER students are promoted to the next grade with new teachers.  

Statewide results for Spring 2014

The below information demonstrates the significant decline in student test scores since the implementation of PA Core Standards.

Algebra 1:  40.1% of students scored Proficient or Advanced:

Students TESTED	Students PASSED	Students to RE-TEST	% FAILED
203,357	81,470	121,887	59.9%

Biology:  41.4% of students scored Proficient or Advanced:

Students TESTED	Students PASSED	Students to RE-TEST	% FAILED
164,757	68,215	96,352	58.6%

Literature:  52.4% of students scored Proficient or Advanced

Students TESTED	Students PASSED	Students to RE-TEST	% FAILED
111,612	75,076	68,222	47.6%

Below are the statewide results for RE-TESTERS (students who tested one or more times).  The students below who did not pass will take a remedial class and re-test AGAIN for the second or third time and/or take the Project Based Assessment.  The students have already passed the class.

Algebra 1 80,591 students re-tested 21.8% passed 63,035 students to re-test AGAIN

Biology  40,7615 students re-tested 11.7% passed 42,061 students to re-test AGAIN

Literature  31,686 students re-tested 20.2% passed 25,273 students to re-test AGAIN

The PA Core Standards have resulted in a significant amount of lost instructional time, an increase in test prep and a significant decline in student test scores and are a burden to teachers and school administrators.

2015 Planned Opt Outs for PSSA and Keystone exams

as of 3/26/15

Issues related to the implementation of PA Core Standards as many parents are looking into opting their children out of the PSSA and Keystone testing.

Parents across the state have been told that weren't ALLOWED to opt out of the PSSA or Keystone testing.  This information is being told to school districts verbally per the PDE during Test Administrator Training.

This is from one participant in the Test Administrator Training:
The misinformation that I received was indeed verbal and not written and took place during a question and answer period for training in the administration of the assessments. The information was that a parent must list "specifically what part of the test or what in the test is a religious objection.

This is from one parent:
The principal replied with. " I must submit in writing to the assistant superintendent the specifics of your religious grounds to refuse the test. The PDE requires parents to be specific in their refusal."

THE KEYSTONE GRADUATION REQUIREMENT HAS NEGATIVE CONSEQUENCES (EVEN FOR STUDENTS WHO TEST PROFICIENT ON THEIR FIRST TRY)

Although there has been a great deal of focus on students who have problems PASSING the Keystones, it must be noted that even students who DO test proficient on the first try are being harmed by the Keystone graduation requirement debacle.  Excessive concerns about the effect of test scores on school performance and teacher evaluations encourage teachers to “teach to the test,” an educationally unsound approach to learning that often hinders a lasting understanding of the course material.  It’s called “cramming!”

EXAMPLE: A middle school relative of mine, who incidentally is labelled as “gifted,” took the Algebra I Keystone last year and tested Proficient.  That SHOULD mean that he has a good understanding of the algebra, right?  NO…WRONG!  This child who tested “Proficient” in Algebra I is currently taking geometry in his middle school and struggling.  The teacher attributes his poor performance in geometry to his lack of understanding of crucial algebraic concepts.  His parents are now paying over $300/month to have him tutored in a learning center to fill in the huge gaps in algebraic understanding that he will need in his subsequent math courses.  Because this student is bright, the weeks of “teaching to the test” enabled him to test Proficient on the Keystone, but unfortunately, his “proficiency” was only temporary.  Perhaps if the Algebra I teacher had been able to focus more on ensuring that her students thoroughly understood essential algebraic concepts rather than having to spend inordinate amounts of time reviewing specific examples of Keystone-type problems (“teaching to the test”), this vast expenditure of money on tutoring wouldn’t be necessary.
 

Testimonials

My son's school said they have to pass all keystones or not graduate! He passed all but algebra. He has to be in a remedial Keystone math class and not only pass class but pass each part of the keystone test given in small parts. He passed 2. They said the PDE told them they have to pass them for graduation because it's part of the No Child Left Behind Act! Right now my son has a grade of 70. He normally has a D and is barely passing. His teacher is trying to help them pass the best he can but my son is so stressed out!  He is in graduating in 2015.

My daughter is in 10th. She took the Keystones last year and scored Basic.  She passed the Biology Class. This year, they required her to complete a 2 semester Keystone Biology class prior to taking the Keystones. Again, she passed the class. Her score on the Keystone was 1480. Now, she is telling me that she will be unable to attend Vo tech because she will have to take yet another class, to complete the Keystone project based assessment class. I am DONE. Done with all this testing.  I am so fed up with this testing. I have 2 older children who didn't have to go through all this. The more it gets implemented, the worse it is. My daughter gets hives when she starts thinking about testing. She is an average student, but she is also getting fed up with the tests.

I talked to the guidance counselor regarding the repeat class. Told it was just to help her understand biology better. It was a struggle the first round because of the teaching style of the teacher. So, they put her right back in with the SAME TEACHER to test prep. I now realize that is all this class is. I also found it interesting that my daughter was afraid to show me the test prep book. She said they were told not to show it… I told her...give me the book. I have it. They have set these kids up for failure. I am not happy. She has struggled with self-esteem issues and some classes. And this just makes her feel stupid and worthless. Not having it...for her or any other child.

 I have a friend whose son just took the Algebra test for the 3rd time and failed and now they are saying in order to graduate he must complete the Project? However, I think there is a way to also opt-out of this, correct? MANY THANKS.


Hi in opting my child out of pssa's. The word in our district is the following: "your child cannot be at school during PSSA testing if they are not taking the test. You will have to provide transportation to get your could to school after testing is done for the day. I asked last year". Isn't this discrimination? Before sending my letter to opt out, I want to make sure I know my right. As a working parent having to drive my daughter in late for nine days would be troublesome. I was under the impression children go to the library during testing and are provided with additional learning activities. Can you help? Thanks!

My son is a senior and I was told if he didn't take the tests he couldn't graduate... Well he had an A for his final grade in biology so I didn’t push the issue of not taking the test... We'll guess what he didn't pass and he always does his best... When I was talking to the guidance about it she said almost 70 percent of the grade didn't pass!!!!!! My son is very bright... These tests r just wrong.... Just wait to see what they make u have to do when u don't pass… It's nuts.... For the 70 percent of the whole senior class basically have to do more tests and more tests and basically almost have to take the class over when u already passed it with flying colors.... Because they said they have to prove to the state that they taught him bio... We'll doesn't an honest A. in the class prove that.? My younger son is only a b student (and I'm ok with B). But wow I'm sooooo worried ...things need to change …

How do you opt out? My daughter just found out she didn't pass the algebra (4^th time trying) or bio (2nd time). She asked me why she has to go to school since she isn't going to graduate. They also discovered in her bio class that there are things on the bio that they were never taught because it is from a class that juniors take. This is what her bio teacher told her.

My daughter is a junior and will be graduating in 2016. The school district is using the Keystones as a graduation requirement for her class. My daughter has passed the Biology and English Keystones but not the Algebra I Keystone after three attempts. In junior high the school district has an Algebra class A and B. A is taken in their 7th grade year and B in their 8th. My daughter was first tested for the Algebra I keystone in her 7th grade year when she only had a half year of Algebra. She did not pass. She finished Algebra I in her 8th grade year but was not tested. The school districted waited a year later, when she was in 9th grade and had been taking Geometry to test her again. After being out of Algebra for a year she was not able to pass the test again. She took Algebra II in her 10th grade year. The school district did not test her. Over the summer I was told that any student who did not pass would have to take remedial math their senior year. I was not happy about this. My daughter has always been in the more advance math classes for her class and has earned all A's. She is currently in pre-calculus receiving A's. Now she may have to take a remedial math her senior year. The school district was planning on letting some of the juniors take the test in January. My daughter was not going to be included in that testing. After fighting with the school district they allowed her to take it. I just found out the other day that she did not pass again. They will allow her to take it a fourth time but she is getting very frustrated and discouraged.

My daughter and I first heard of the Keystones last year when she was in 8th grade. My daughter is an honors student and the National Junior Honor Society.  She came home very upset, telling me that she had to take a test and if she failed she couldn't graduate. I said: "I think you're being dramatic. They are not going to give you a test in 8th grade, at 13 that determines whether you graduate or not. If they did that, then everyone who fails would have no reason to finish school. But I'll call and talk to your guidance counselor." When I called, I was shocked to find out that that is exactly the case. How ridiculous can you get? Telling a 13 year old that they won't graduate if they don't pass a test 5 years before graduation! She took the test and failed the algebra portion. She was devastated. Crying because now this score was in her permanent record. She had never failed anything in her life!  Since she failed the test, they put her in remedial algebra, which is kind of baffling, considering she passed the class with a "B". 1 test changed the course of her schooling. Maybe she was having a bad day. Maybe she wasn't feeling well. Maybe she was so stressed out about such a high stakes test that she just bombed it! However, none of those things are taken into consideration once the scores are in. Since then, I have opted her out of the Keystones, and then withdrew her, as well as the rest of my kids from school altogether. How can I trust the schools to adequately educate my children, when they base the course of their schooling on sporadic high stakes testing? They can keep their tests, I'll keep my kids.

Keystone Exam Information and Q&A

Questions and Answers Regarding Pennsylvania’s Keystone Exams

From the year 2017 and thereafter, testing Proficient or Advanced in three Keystone exams (Algebra I, Language Arts, and Biology) will be a high school graduation requirement for all PA students.  There is a great deal of information on the PDE’s web site concerning these exams.  The web site: http://optoutpa.blogspot.com/ also provides invaluable and detailed information on both the Keystones and PSSA’s.  The questions answered below should be helpful for understanding some aspects of the Keystones that are not normally publicized.  They specifically focus on the Algebra I Keystone, but answers are similar for the other Keystones.  Thank you to John Weiss of the PDE for providing many valuable insights and clarifications.

Question: Can a student graduate from high school if s/he is unable to pass a Keystone?  What are the effects on students and school districts from circumventions of this graduation requirement?

Answer: Yes. There are avenues that the PDE has put in place that enable students who cannot pass the Keystones to graduate; however, these circumventions take an insurmountable toll on both students and school districts.  Students can retake the Keystone any number of times to try to test Proficient.  (Note that this is quite lucrative for the test-making companies!)  Once they've taken it twice and still can’t test Proficient, students can then take what's called a "Project Based Assessment”[1]. However, the time involved in these processes and the various types of remediation that frequently occur simultaneously, preclude students from taking courses that are more essential to their chosen career goals, thus inhibiting them from reaching their individual maximum educational potentials.  (Children aren’t cookies made from the same cookie cutter. “One-size-fits-all” does NOT apply!)  The mental and physical stress inflicted on many students during this process is well-documented and would be avoidable by abolishing this requirement.  Last, but certainly not least, is the enormous fiscal impact on school districts, who are already financially strapped.  The huge expenditures required by school districts to comply with this unfunded mandate could be utilized in far more effective ways in improving our educational system.

Question: What types of questions are on the Algebra I Keystones?

Answer: There are MULTIPLE CHOICE questions (fill in the blank, and graded by machines), and SUBJECTIVE questions (AKA “constructed response”) which are graded by scorers and for which students have to show their work.

Question: How are the Algebra I questions proportioned between MULTIPLE CHOICE and SUBJECTIVE (AKA “constructed response”)?

Answer: There are 36 multiple choice (1 point each) and 6 subjective questions (4 points each) for every exam.  This adds up to a total of 60 points.  The subjective questions (24 pts.) comprise 40% of the exam score.

Question: Who grades the subjective questions?

Answer: Temporary employees hired by Data Recognition Corporation (DRC - the company contracted to handle PA’s assessments) hand-grade the six, 4-point subjective questions which account for 40% of the overall score (Algebra I).  Scorers are provided with rubrics for this task.

Question: How many scorers grade each subjective question?

Answer: 90% of the subjective questions are graded by only one scorer, 10% by two scorers.  

Think of this… 90% of the subjective questions that determine 40% of the final score on this high stakes test are graded by only one scorer!

Note: For such a high stakes test, it would seem to be essential to have two graders grading every subjective question, and if the scores deviate, to bring in a third, particularly because, as will be indicated, there are legitimate concerns as to the quality and credentials of the graders.  While this would be expensive, it would seem to be the fairest way to assure accuracy in grading.

Question: What are the qualifications of the scorers of the subjective questions?

Answer: Although the PDE stands by, and supports what they SAY are appropriate qualifications and training of scorers, there are legitimate concerns regarding the manner in which they are hired by DRC.  First of all, these scorers are temporary employees that earn between $11-14/hour.  Their only required qualification seems to be a four year degree, which could be virtually meaningless, considering what is posted on-line by interviewees commenting on the interview process.  Although the PDE insists that “numerous quality control measures are implemented to monitor and maintain accuracy and reliability” during the scoring of the subjective questions, the comments below [2]: certainly don’t allay the legitimate concerns regarding the suitability of scorers who will be determining 40% of the grade in such a high-stakes test.  Four of these comments are:

       

·      “Brought in batches to be tested for middle school intelligence in a computer lab. Taken in for individual interviews just to basically confirm we had a pulse, didn't forge our diplomas, and weren't a threat to others or ourselves. Hired on the spot. (Sample Interview Question – x+1=2)”

·      “We were asked to write and(sic) essay and complete some math problems to demonstrate proficiency. We were then interviewed in the order we finished. The interview was short and just asked basic questions about why you were qualified for the job. You were also asked how you would be able to handle a job that required you to sit and read responses for eight hours.  Interview Question – The questions were very basic. You just have to prove you have the credentials and focus to do the scoring.”

·      “Come in to the office, listen to a presentation, and take two written assessments. One is a pre-algebra level math test, and the other is a writing sample, my prompt was ‘describe your best achievement in the last two years’. After turning in your assessments, you talk one on one with the HR rep for a short (10 minute) interview where she basically wants to see if you will be comfortable with the close quarters of the working environment, the repetition and monotony of grading, and if you can be consistently impartial. If you're smart you just say yes. Interview Question – Honestly, there was nothing you couldn't answer easily off the cuff, this is not applying to Harvard.”

·      “It was a group interview. They would then take each person individually to a room and ask them what their area of expertise was, such as English, Math or Social Sciences. They would then place you in whatever room best fit your expertise in order to score the state's standardized test scores. Interview Question – Where do you see yourself in five years?”

Question: After the Keystone is scored and results have been obtained, can a parent review the test?

Answer: Yes, upon approval by the PDE.  For a processing fee of $75, the parent can view the student’s Test Booklet, but cannot see the questions to which the student’s answers refer. 

Question: Can a parent have their child’s test rescored?

Answer: Yes, they can, for a fee of $75. According to John Weiss of the PDE, “A parent would request a rescore following the same procedure as a request for the student’s response.  The rescore process costs $75.00 and includes a manual verification that the multiple choice items were captured correctly, as well as a rescoring of all of the student’s constructed responses.  Each step is verified by a second person.  The results of the rescore are documented and forwarded to the school/district.”

Question: What are the different sections and raw score point distributions for the Algebra I Keystone?

Answer: There are TWO MODULES, each of which is worth 30 points.  (Remember that the total number of points for the raw score is 60, with 36 points for multiple choice questions and 24 points for the subjective questions.)  Below is the distribution:

Module 1: Operations and Linear Equations and Inequalities: (30 pts.)

Operations with Real Numbers and Expressions (9 pts.)

Linear Equations (10 pts.)

Linear Inequalities (11 pts.)

                      Module 2: Linear Functions and Data Organizations: (30 pts.)

Functions (10 pts.)

Coordinate Geometry (10 pts.)

Data Analysis (10 pts.)

Question: What are the scale scores that show the ranges that indicate whether a student has tested Below Basic, Basic, Proficient or Advanced?  Note that Below Basic and Basic are not passing.)

Answer: The following chart shows the scale scores.  These scale score ranges have been set arbitrarily; they range from 1200-1800, just as SAT scores arbitrarily range from 200-800.  .  While the raw score to scale score correspondence (AKA “cut scores”) might change slightly from one administration to another, the scale score ranges designating the cutoffs (shown below) remain the same for every administration of a Keystone and do not change from year to year.  In other words, 1500 and above is always passing for each of the three Keystones.

 Keystone Exam Scale Score Ranges

Question: Since there are two modules, can a student score Proficient on one module and Basic on another and still test Proficient overall?

Answer: Yes, if the raw scores of the two modules added together produce a total raw score that corresponds to 1500 or above on the scale score.

Question: If a student tests Proficient on one module and Basic on the second module, does s/he have to redo both modules in a retest?

Answer: No, the student only has to retake the module that s/he did not pass.  However, Mr. Weiss of the PDE stated: “It may be in the student’s interest to retake both modules since a higher score on the passed module with no improvement on the other module could result in a total test score (highest total test scale score to date) that is Proficient or above.”

Question: How are the raw scores (60 pts maximum) converted to the scale scores ranging from 1200 – 1800?  Is the raw score to scale score correspondence (AKA “cut scores”) the same for every test administration?

Answer: According to Mr. Weiss, the raw score to scale score correspondence is not the same for every test administration.  He explained: “Raw scores or percent correct scores cannot be used to compare students’ achievement across administrations because they depend on the difficulty of the tests.  Raw scores are converted to scale scores through the process of equating and scaling.  Equating is a statistical process used to establish comparability between scores from different test administrations.  Equated test scores can be used interchangeably even though the test forms consist of different items.”

Question: Shouldn’t we be concerned that the raw score to scale score correspondence (AKA “cut scores”) could be manipulated and subjectively placed in order to control the percent of students who pass?

Answer: According to Mr. Weiss, this does not happen.  He said: “Keystone technical reports, which include raw to scale conversion tables, are posted online on PDE’s website. Within that document, you can see that the equating and scaling procedures are reviewed and approved by Pennsylvania’s technical advisory committee (TAC), strictly followed, and replicated prior to reporting.”  Also, according to PA School Code Chapter 4.51b (g), the PDE “will use widely-accepted psychometric procedures to establish the cut scores. Cut scores shall be presented at a public meeting of the Board for its review at least 2 weeks prior to scheduled Board action on the cut scores.”

Question: What company handles the testing procedures for Pennsylvania?  What are the costs?

Answer: The testing company that Pennsylvania contracted with from 2009 – 2015 to create, distribute and score the Keystone Exams and the PSSA's, is Data Recognition Corporation or DRC.  As was discussed previously, DRC hires the temporary scorers to score the subjective portions of the exams.  The 2009-2015 contract with DRC involved $200+ million.  It was rumored that PA was going to change to a company called Measured Progress after the DRC contract ran out, but they have extended DRC’s contract through 2016 instead.

Question: How much time do students have for the Algebra I Keystone?

Answer: Estimated time for multiple choice questions is 1.5 minutes/question; estimated time for subjective questions is 10 minutes/question.  The estimated total test time is therefore approximately two hours.  However, students can take as much time as is necessary to complete the exam; it is essentially “untimed.”

Question: When are the May Keystone results available to school districts and to students?

Answer: School districts receive scores by the second week in July, but many districts do not make these results available to students until the first week in September.  This delay can be problematic since scheduling often has to be readjusted for students who have not demonstrated proficiency in a given Keystone, either because they might have to retake the course, or because they will have to adjust their schedules for remediation. 

Question: What information is provided in the report that is given to the student to show his/her results on the Keystone exam?  Is it helpful in pointing out his/her specific strengths and weaknesses in the subject material?

Answer: A sample report is given below.  It is very sketchy and provides negligible feedback for enabling students and parents to understand the student’s strength and weaknesses in various areas.  It gives only the scale score for the two modules; there is no breakdown for the six sub-sections (3/module).  This unfortunately is an indication that the educational aspects of the test results are secondary to the primary purpose of the test, which seems to be its use in evaluating teachers and schools.  Mr. Weiss has stated that: “Raw score by Assessment Anchor is provided for each student receiving a performance level. This detailed information is provided in the District Student Data Files (DSDF).   Along with the DSDF, a file layout is posted that has the total points possible by Assessment Anchor. Similar to the Performance Summary by Assessment Anchor, this breakdown is based on the current test administration.”  If this is the case, it is hard to understand why the student’s report doesn’t include these data.

Prepared by Joanne Yurchak (jyurchak@comcast.net) with the help of Dawn Sweeney (http://optoutpa.blogspot.com/), and with some information provided by John Weiss of the Pennsylvania Department of Education.

---

[1] http://static.pdesas.org/Content/Documents/Keystone%20Project%20FAQ.pdf

---

[2] http://www.glassdoor.com/Interview/Data-Recognition-Corporation-Interview-Questions-E233679.htm