US20070105080A1 - Application of multi-media technology to computer administered vocational personnel assessment - Google Patents

Application of multi-media technology to computer administered vocational personnel assessment Download PDF

Info

Publication number
US20070105080A1
US20070105080A1 US11/645,888 US64588806A US2007105080A1 US 20070105080 A1 US20070105080 A1 US 20070105080A1 US 64588806 A US64588806 A US 64588806A US 2007105080 A1 US2007105080 A1 US 2007105080A1
Authority
US
United States
Prior art keywords
subject
multimedia
computer
stimuli
skills
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11/645,888
Inventor
Michael Hersh
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US08/829,488 external-priority patent/US6030226A/en
Application filed by Individual filed Critical Individual
Priority to US11/645,888 priority Critical patent/US20070105080A1/en
Publication of US20070105080A1 publication Critical patent/US20070105080A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00Electrically-operated teaching apparatus or devices working with questions and answers
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/16Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00Electrically-operated educational appliances
    • G09B5/06Electrically-operated educational appliances with both visual and audible presentation of the material to be studied
    • G09B5/065Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00Electrically-operated teaching apparatus or devices working with questions and answers
    • G09B7/02Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/40Detecting, measuring or recording for evaluating the nervous system
    • A61B5/4029Detecting, measuring or recording for evaluating the nervous system for evaluating the peripheral nervous systems
    • A61B5/4035Evaluating the autonomic nervous system

Definitions

  • This invention is related to interactive multi-media applications in psychological testing for vocational assessment, which evaluates a job candidate's ability to maintain task focus under pressure, rather than only inferring performance based upon selective input from a candidate.
  • My above noted prior applications are incorporated by reference herein.
  • My prior patent applications are concerned with applications of multimedia technology for psychological and educational assessment tools.
  • My prior applications are an improvement over conventional psychological and educational assessment tools which have been used for the measurement and evaluation of abilities, aptitudes, personality characteristics and so forth, such as the intelligence quota (IQ) test dating back to 1904 of the French psychologist Alfred Binet that would identify slow learners.
  • IQ intelligence quota
  • CAI computer-assisted instruction
  • CAI computer assisted instruction
  • courseware is provided for teaching presentations in various learning areas.
  • CAI computer assisted instruction
  • the subjective factors of reliability, validity, bias, and standardization are seldom of central importance because of the objective teaching nature of the CAI courseware.
  • assessment tools are rigorously field tested (i.e. “normed”) for reliability and validity, and their mode of administration is precisely stipulated, and any deviation may violate the conditions under which it had been normed.
  • CAI Computer Assisted Instruction
  • CAA Computer Assisted Assessment
  • CAA Computer Assisted Assessment
  • Scoring of Computer Assisted Assessment (CAA) is sophisticated with raw scores, standard scores, sub-tests, age equivalents, and basals and ceilings.
  • the testing rules of Computer Assisted Assessment (CAA) are rigid with no deviation allowed.
  • My prior patent emphasizes that a clinical psychological tester can use the Computer Assisted Assessment (CAA) with its attendant multimedia tools, in the area of question sequencing and scoring related to establishing the critical range of the tested subject, and that the subject's basals and ceilings are determined by runs of right or wrong answers within a particular sequence.
  • CAA Computer Assisted Assessment
  • My patent also discusses a method for reducing bias, such as social class prejudice or the assumption that shy or disruptive children are learning impaired.
  • My patent also overcomes that fact that some clinicians may be very proficient in their respective fields, but that they may not speak clearly, which causes confusion. My patent reduces the probability that a tested subject will get a question wrong just because he or she doesn't understand the examiner, especially in English as a Second Language (ESL) subjects.
  • ESL Second Language
  • My patent also discusses the option that the multi-media computer has a linguistic adaptation means, such as a sound card and data associated therewith, which can produce sounds, such as test instructions, in the most appropriate language and dialect of the test taker, such as a student, so that the human tester can provide the human subject with instructions for responding to the psychological evaluation.
  • a linguistic adaptation means such as a sound card and data associated therewith, which can produce sounds, such as test instructions, in the most appropriate language and dialect of the test taker, such as a student, so that the human tester can provide the human subject with instructions for responding to the psychological evaluation.
  • the computer tallies and records the subject's responses and analyzes the human subject's recorded responses. Afterward the computer automatically prepares a report based upon the human subject's recorded responses, and transfers its report, such as by transfer to a computer-readable diskette or by printing to a printer.
  • the psychological evaluation of the human subject is provided by at least one software program stored in a randomly accessible non-volatile computer memory, such as the computer's hard drive, for providing at least one type of psychological evaluation.
  • the at least one software program further includes a plurality of discrete audible and/or visual segments such as questions to be responded to by the human subject being evaluated.
  • the discrete audible segments include audibly reproducible signals stored in a randomly accessible non-volatile computer memory for producing sounds to be responded to by the human subject being evaluated, such as spoken phrases to be responded to by the human subject.
  • the spoken phrases are selected from a plurality of user-selectable languages and/or dialects.
  • My patent also discusses the fact that to assist in evaluating temporal time oriented tasks, such as allowing a person decide which of a plurality of moving objects, such as balls, moves fastest, or the calculated destination thereof, optionally the discrete visual segments may include computer-generated motion for psycho-motor evaluation of human subjects.
  • My patent disclosed typical video display screen's views of various multi-picture panels used in the computer assisted psychological assessment.
  • My drawings also disclosed video displays using screen panels, optional animation and voice response modes.
  • My patent also provides for an input response means 23 which is used to input communicated sensory data from the human subject to multi-media computer to conduct a psychological evaluation of the human subject.
  • My patent also shows a psychological assessment session, using a multi-media system to give the same type of test as is done in conventional human-to-human evaluations.
  • My patent discloses how a tested human subject sits facing a computer screen on a video display terminal of a multi-media personal computer, wherein the subject uses pointing device to indicate his or her answers.
  • My patent also discusses various multi-media computer components, such as a mother board containing a microprocessor and other modules and support electronics, with optional extra random access memory (RAM), a hard drive, a floppy drive using removable media (diskettes), a CD-ROM with a removable compact disk (CD), a sound board and a variety of sockets or ports to support a speaker, a microphone, a display, a mouse port and a keyboard socket.
  • RAM random access memory
  • CD-ROM with a removable compact disk (CD)
  • sound board and a variety of sockets or ports to support a speaker, a microphone, a display, a mouse port and a keyboard socket.
  • My patent gives a non-limiting example of an appropriate multi-media computer, such as a Packard Bell model AX3510, which has 8 MB RAM, a 1000 MB hard drive, Quad speed CD-ROM, stereo speakers and an Intel 100 MHZ Pentium processor.
  • a Packard Bell model AX3510 which has 8 MB RAM, a 1000 MB hard drive, Quad speed CD-ROM, stereo speakers and an Intel 100 MHZ Pentium processor.
  • Computer Assisted Assessment tools may run on a 33 MHZ '386 class processor with a 100 MB hard drive, single speed CD-ROM and 4 MB of RAM, or a notebook computer having similar capability with integral liquid crystal display (LCD).
  • a 33 MHZ '386 class processor with a 100 MB hard drive, single speed CD-ROM and 4 MB of RAM, or a notebook computer having similar capability with integral liquid crystal display (LCD).
  • LCD liquid crystal display
  • My patent discloses that any type of pointing device can be used; such as, for example, a mouse, track ball, video game control, joy stick, or even a finger when used with a “touch screen”, as well as other display technologies such as plasma displays or Texas Instrument's micro-mirrors, which can support 3-D presentation displays, which are most appropriate in psychological assessment tests.
  • My patent shows that educational and psychological assessment test content is prepared using state-of-the-art multi-media authoring tools combining audio, picture and text sources, which improves over the manual administration of these psychological assessment tools, so that the automatic nature of using a computer precludes bias in test administration.
  • My patent shows that multi-media technology can selectively first show an image with a plurality of objects, and then can eliminate a number of the presented objects to focus the tested subject on an appropriate response.
  • My patent also shows that animation can be used in creating new measures dealing with temporal issues, such as movement of objects, which is also appropriate in my present invention which, among other features, can virtually simulate a vocational environment of a job candidate.
  • animation can also test cognitive skills in predicting events related to the virtual work environment.
  • a multi-media computer is ideally suited for assessment tools that require the use of timing, to replace assessment evaluations requiring the use of a stopwatch by the clinician to either time the presentation of materials or the period for timed responses (or both).
  • My patent also shows that a multi-media computer may have voice recognition software of proper quality that it can reliably recognize the utterances of a tested subject and that a microphone can be used to field the responses and determine if an answer is right or wrong, or if more than one response has been given.
  • Such state-of-the-art of voice recognition software is disclosed in software of Syracuse Language Centers, which run on a 386 -class computer with 4 MB of RAM and are capable of analyzing the pronunciation of words automatically.
  • My patent also discloses that different keys on a keyboard or pointer can also be linked to specific verbal responses or prompts. For example, if the test subject gives an ambiguous answer, the computer can audibly say, “Explain what you mean.” In the present invention, the computer can evaluate stressful assessment situations, by uttering critical statements, such as, “You're not working fast enough,” and then can use biofeedback monitoring to evaluate the subject under stress.
  • My patent discusses the use of appropriate language dialects, as well as an “assisted voice recognition” system which uses a keyboard and no microphone.
  • keys on the keyboard are mapped onto special functions by the testing software, wherein responses of a tested subject may be responded to with an audio feedback such as, “Please respond with only one answer” every time one of the “Invalid” keys is pushed.
  • My patent also discusses how incorrect answers can be responded to if answered incorrectly.
  • the test can be automatically terminated if the test subject never gets beyond an initial stage. After the test, a raw score is updated, the basal/ceiling analysis is performed, the test score is updated, the sequence is changed if necessary, and the subsequent following question is presented (or the test is ended).
  • This general flow can be used with pointing as well as verbal answers. It is understood that other logical flows can be used to achieve an equivalent result without deviating from the scope of this invention.
  • My patent also demonstrates various video display screens displaying multi-media techniques in modeling appropriate responses for sample questions.
  • the drawing Figures show screens with pictures of objects wherein the tested subject is asked to point to one of the objects and if the answer is incorrect, the respective display screen shows the image corresponding to the correct answer. By not showing the other two images in the same screen in the present invention, the student's attention is focused on the correct answer.
  • My patent also shows a typical example known as a CREVT-CA program, which shows how the method and system of the present invention is used in one illustrative example.
  • a sample psychological/educational assessment test is run in a computer program designated as the “Comprehensive Responsive and Expressive Vocabulary Test-Computer Administered” (CREVT-CA).
  • the user double clicks on the CREVT-CA icon in the CREVT-CA Program Group. If the user has installed the program in WINDOWS 95®, from the start menu, the user selects “Programs, CREVT Program Group”, with the CREVT-CA icon. After the opening title, the Main Menu appears. Six choices on this screen are available to the user, such as:
  • the user points and clicks on the box next to the operation the user wishes to bring forth.
  • My patent also shows how a typical program, such as a CREVT-CA is used to administer a test to an individual.
  • the user can choose to administer both the Receptive and Expressive sub-tests or individual sub-tests.
  • the user selects an appropriate language and dialect of the test subject, such as a child, such as “French-Quebecois” vs. “French-Haitian Creole”, or “English-Australian” vs. “English-Southern American” and the like. When this is done, the user clicks on the “OK” button to continue.
  • an appropriate language and dialect of the test subject such as a child, such as “French-Quebecois” vs. “French-Haitian Creole”, or “English-Australian” vs. “English-Southern American” and the like.
  • the user chooses which form of the test the user would like to administer—Form A or Form B. If the user is administering both sub-tests, the form selection applies to both sub-tests.
  • the testing begins with the Receptive subtest.
  • the subject is seated in front of the computer with the mouse in hand (in other test environments, the examiner may input information from the subject, who is tested in a hands-free environment).
  • a voice in a pre-selected language and dialect pronounces a word while the subject views the pictures.
  • the subject simply uses a pointer, such as a computer mouse to point and click on a picture to indicate an answer.
  • the program automatically moves on to the next set of pictures.
  • the user can still have the CREVT-CA compute the standard scores, percentiles, and age equivalents for both sub-tests together.
  • the user simply clicks on the “yes” button to choose to score the Expressive subtest as well.
  • the CREVT-CA program asks the user to enter the raw score of the Expressive subtest and then computes the Receptive, Expressive, and Composite scores.
  • the user After the Receptive subtest is completed, the user needs to enter the user's password to be able to view the scores (the password is initially set to “CREVT”).
  • the screen displays a graph of the percentage of correct answers per set of pictures. This graph can be printed by clicking on the “OK” button next to the word “Print” at the bottom of the screen.
  • the Expressive portion of the test is given on the computer after the Receptive portion or by itself.
  • the computer gives the instructions orally and asks the subject to define a series of words.
  • the Expressive Subtest automatically ends when either a ceiling is reached (i.e., when the subject misses three consecutive items) or when the last (25th) item is administered.
  • the examiner when the testing is finished for a particular subject, the examiner enters a password (the password is initially set to “CREVT”) to view a comprehensive report of the test results on the computer screen. The examiner can then print out a hard copy of the report by clicking on the “PRINT” button at the bottom of the scores screen. If the Expressive subtest was administered and some information was saved in the “SUBJECT'S REPLY” text boxes, the examiner has the option of also printing out those replies.
  • CREVT the password is initially set to “CREVT”
  • the CREVT-CA program also allows the user to graph the standard scores obtained by the subject, along with up to seven standard scores with means of 100 and standard deviations of 15 obtained when using other tests with the subject.
  • the user chooses to view a previous record option from the Main Menu.
  • the user enters the name of the subject whose record the user wants to see.
  • the user can search for a record by last name only or by first and last names.
  • the user can also choose to view the database record by record.
  • Once the desired record is displayed on the screen, the user can choose to view either the subject's scores or the subject's responses to the Expressive Vocabulary subtest.
  • the user can also print out hard copies of either of these items.
  • many of the aforesaid multi-media applications can be specifically used in the field of vocational psychological assessment, to ascertain whether a tested subject is suitably fitted for a particular job. Not only can the person's cognitive aptitude be determined, but the tested person can be monitored while subjected to non-stressful and stressful situations, to simulate the rigors of a particular job or the person's suitability in general for a particular job.
  • U.S. Pat. No. 4,683,891 of Cornellier discloses a computerized employment training using biofeedback monitoring to enable a subject to increase his tolerance of stress and enable him to perform tasks under more stressful conditions, using goal oriented cognitive tasks.
  • test performance, interpretation and scoring criteria which evolves over time to reflect changes in the population subjected to the various test modules to establish norms more representative of the current population.
  • test administrator can modify the module selection, the pass/fail limits, and the weights assigned to each module for a particular job title or for a particular individual, in a variety of job performance modules.
  • the present invention provides a battery of tests which are administered to an individual by a computer using multi-media features.
  • This normed testing can be used to assess the suitability of an applicant for employment, although other versions of the test emphasizing the stress tolerance module can also be used for screening candidates for high-stress or high-risk mission critical assignments.
  • the primary objective of this invention is to reliably and accurately predict real world success using an easily administered rapid assessment technique.
  • an important feature of this assessment system is to maintain a database of field experience to continually refine the norms to insure that they are representative of the current population of individuals subjected to these tests. Updates of these norms are distributed to customers.
  • the actual software product is delivered to customers as portable media such as CD-ROM, diskettes, DVD or similar computer input media. It is also available for download to a test administrator in secure fashion from an Internet web site. Direct use from a web site on the Internet or on a company intranet is another option for test delivery.
  • the multi-media software may have an interface utilizing one or more computer input hardware devices such as a mouse, a keyboard, a touch pad, a microphone, a camera, physiological sensors or any other computer input device.
  • the software is delivered to a test administrator as portable media such as CD-ROM, diskette, ROM, non-volatile RAM, flash media, DVD or other computer compatible input media.
  • the software may be also be downloaded from the Internet, where access is obtained through the Internet.
  • the tests may be administered through the Internet and the related administrative functions may be performed through the Internet.
  • the Internet there may be secured communication, wherein the secured communication is implemented with one or more security technologies, such as an internet protocol security, a secure socket layer, a private key encryption and/or a public key encryption or may utilize a VPN or other form of secure tunneling technology.
  • the present invention includes a personnel evaluation system with one or more multimedia computers for sending and receiving audio and visual stimuli, each connected to respective central processing units to administer a pre-employment evaluation, a pre-promotion evaluation, a pre or post training evaluation, a personnel development/leadership training evaluation, or other form of employment evaluation application evaluation and to perform an analysis of input data.
  • the software application stores and evaluates all input data and includes one or more modules to monitor and re-scale non-truthful responses of prospective employees, wherein one or more interfaces constantly monitor a physiological state of a subject. These interfaces induce stress inducing audio and visual stimuli and administer the evaluations with race, gender, ethnic, and cultural neutral stimuli.
  • the interface may interact with the subject in their native language, and dialect.
  • the interface assesses various aspects of the subject which might affect work performance, such as intelligence, interest, values, emotional status, mental health, neurological impairment, education, social apperception and aptitude of the subject being tested.
  • the system can detect for untruthful responses and re-scale values of questions depending on the type of test and detect physiological changes in the subject.
  • physiological factors which are monitored by biometric sensors include measuring one or more of the following: galvanic skin responses, muscle tone, facial expressions, pupil dilation, heart rate, EKG electrocardiogram responses, blood pressure, thoracic respiration, abdominal respiration, saliva composition.
  • a menu derived from subject population statistics is presented to the test administrator by the software which displays the suitability of the various test modules, the pass/fail limits, and the weight assigned to each module, all with respect to a particular job title. The administrator can modify these suggested parameters to tailor the screening to his department needs.
  • the job performance modules include tests for Integrity, Aptitude, Loyalty, Stress Tolerance, Memory, Social Awareness, Communications Skills, Performance Efficiency, Distractibility, and Impulsivity.
  • Aptitude sub-tests include Visual Closure, Visual Analogies, Categorization, Information, Personality, Interests, and Values.
  • the computer administered multi-media stress tolerance test module compares a subject's performance on a visual matching task under two conditions, non-stress and stress.
  • the visual matching task uses pictures of 3-D shapes and objects that are carefully selected to avoid bias to different ethnic groups or socio-economic status. Stress factors used are audio coaching telling the user to hurry, chiding the user for making errors, and short time limits for test segments.
  • physiological sensors that actually measure and quantify changes in the autonomic nervous system are attached to the subject. These sensors detect the onset and degree of such changes before, during and after the stress test.
  • Sensors on the subject's chest, arm, and fingers measure heart rate, EKG electrocardiogram responses, skin temperature, skin conductance, muscle tension, respiration rate, facial expressions, eye pupil dilation, galvanic skin responses, muscle tone, facial expressions, thoracic respiration, abdominal respiration, saliva composition and blood pressure and the changes and degree of changes, in all of the above, before, during and after the stress test. Additionally, digital recording of facial expressions and pupil dilation are made. Thus, these changes and the degree of change of these physiological parameters are monitored during the entire test module session. Combined with subject recovery time data, a very comprehensive study of the reaction to physiologically induced stress can be made in a clinical environment. Examination of difference scores for correct matches, errors of commission, and errors of omission provides a rapid and precise indication as to the degree to which an individual is affected by stress as well as the analysis of the likely type of error an individual would make under stress.
  • modules to test memory and distractibility is very difficult to administer in a consistent fashion in any other manner.
  • a computer administered multi-media visual analogies test module uses a combination of audio instruction with presentation of two 3-D color pictures of geometric shapes related in some way followed by a third picture which is related in the same way to one of five choices presented.
  • the shapes are chosen to minimize bias to different ethnic groups or socio-economic status.
  • a visual closure test is efficiently administered as a picture is revealed progressively as it replaces random background elements.
  • a computer administered social apperception test presents an array of high-resolution pictures of people in a variety of postures and facial expressions.
  • the audio presents a spoken statement.
  • the subject is to choose the picture of the person most likely to have made that statement using visual body language and expression cues to pick up the subtle nuances of emotion depicted which reflect that of the spoken statement, as well as the nuances of expression and emotion involved.
  • a computer administered integrity test involves a set of written questions with multiple-choice answers. Subsets of questions with related subject matter are dispersed throughout. The answers to the related questions are analyzed as to response time latency variability to detect attempted deception.
  • a computer-generated and administered vocational assessment system uses virtual reality techniques to simulate a variety of vocational work environments or specific situations to ascertain the best placement of a potential employee. This is accomplished by generating specialized test modules or by substituting virtual reality images and sounds within the format of the job performance and aptitude sub-tests described above.
  • FIG. 1 is a test module menu for a “driver” job title of the multi-media vocational assessment system of the present invention
  • FIG. 2 is a screen image and audio for a Vocabulary test thereof
  • FIG. 3 is a screen image and audio for an Information test thereof
  • FIG. 4 is a screen image and audio for a Categorization test thereof
  • FIG. 5 is a screen image and audio for a Social Apperception test thereof
  • FIG. 6 is a screen image and audio for an Auditory Memory test thereof.
  • FIG. 7 is a sample for a screen image and audio for an Auditory Memory with a Visual Distraction test thereof.
  • FIG. 8 shows audio instructions for a Visual Memory test thereof
  • FIG. 9 shows a screen image and an audio for a Visual Memory test thereof
  • FIG. 10 shows a screen image and an audio for a Visual Closure test thereof
  • FIGS. 11A, 11B , 11 C, 11 D, 11 E, 11 F and 11 G show various sample screen images for a Stress Tolerance Test thereof;
  • FIG. 12A displays audio for a Stress Tolerance test thereof
  • FIG. 12B displays a continuation of the audio for Stress Tolerance Test as in FIG. 12A ;
  • FIG. 13A shows a performance results screen for a Stress Tolerance Test thereof
  • FIG. 13B shows a biofeedback monitoring screen for a Stress Tolerance Test thereof
  • FIGS. 13C, 13D and 13 E show various relaxation phase screens following a Stress Tolerance Test thereof
  • FIG. 14 shows a screen image from a sample for an Integrity test thereof.
  • FIG. 15 shows a screen image for a virtual reality depiction of a work environment.
  • the testing system of this invention When used for pre-employment screening of individuals, the testing system of this invention presents an initial menu screen 1 to the test administrator as shown in FIG. 1 .
  • the name “SAVVY RECRUITER” indicates a trade name of Applicant.
  • Menu screen 1 is identified with a specific job title 2 , which is a “general manager” for this example.
  • the job attributes 3 are presented in a column format to the left and their applicability to this specific job title 2 is presented in “weight” 4 column to the immediate right.
  • a job attribute 3 such as “Aptitude” is weighted as being more relevant than “Memory” for the job title 2 of general manager.
  • a report may be generated by clicking on an option 5 for a report and option 6 for display of relevant test weight.
  • scoring threshold 7 may be displayed to indicate the desirability of the applicant for a particular position such as job title 2 .
  • Time column 8 presents an estimate of the time elapsed.
  • the bottom right portion of menu screen 1 presents similar entries for Aptitude Sub-tests 9 . The test administrator can alter these recommended settings to tailor the test evaluations for the company's specific requirements.
  • FIG. 2 shows an introductory sample screen for a Vocabulary test.
  • a professional voice actor speaks the directions 13 printed below.
  • the subject is prompted with an auditory cue 14 such as “chair”, upon which time he or she should click on the chair 15 picture. If he or she realizes that an error has been made, the repeat symbol 12 can be clicked to get a second try.
  • an auditory cue 14 such as “chair”, upon which time he or she should click on the chair 15 picture. If he or she realizes that an error has been made, the repeat symbol 12 can be clicked to get a second try.
  • a sharpener 16 and a calculator 17 should be indicated.
  • a similarly administered test, “Information,” is shown in the screen image of FIG. 3 , wherein Audio instructions 21 are presented. Then audio statements 22 are presented and the tested subject is to indicate the image of the item which answers the question such as eye 24 , nose 23 or teeth 25 . Button 20 can be clicked to correct an answer.
  • FIG. 4 shows a screen image of a Categorization test.
  • the objective is to select the picture of the item which is not like the others. Audio statements 28 would only be given during sample questions. In this case, the picture of the lion 29 is the correct answer.
  • FIG. 5 is a screen image of a sample question from a Social Apperception test.
  • Audio instructions 33 define the objective as to “choose the person that goes with what you heard”.
  • Button image 32 can be clicked to change an answer.
  • Sample statement 34 goes with image 35 since the statement displays anger.
  • FIG. 6 shows a screen from an Auditory Memory test. Audio directions 38 and examples 39 are strictly an introduction. The actual test presents a stream of numbers using audio. The subject has to enter the digits in the same order by clicking on number symbols 40 . He or she may click cancel button 41 to try again, or signal OK 42 if he or she feels his or her answer was the correct sequence.
  • FIG. 7 is a sample introductory screen from an Auditory Memory with Visual Distractions.
  • the audio instructions, 45 and 46 explain the added visual “clutter” 43 consisting of 3-D images of spheres surrounding the numeric display 40 .
  • the test begins as the sequences 47 are read on the audio stream.
  • Visual Memory test audio directions 50 and 51 are shown in FIG. 8 .
  • Shapes are presented in a sequence one at a time in frame 52 .
  • the subject has to click on the shape images continuously displayed in the same order and then click OK 56 when done.
  • the subject can change his answer once by first pressing CANCEL 55 .
  • the image shown in frame 52 is the same shape as item 53 below.
  • Audio instructions 54 comment on the answer of the subject, and give instructions for further proceedings.
  • FIG. 10 is a screen image of a Visual Closure test.
  • Basic audio instruction 60 is presented for this sample.
  • Explanation 61 is also for illustration.
  • the object image 63 emerges as more of random background 62 is converted to image 63 in a progressive fashion.
  • the subject recognizes the picture as one of the four choices 64 presented, he or she clicks on the matching image. This is a very smooth operation, and the time to answer is accurately noted by the computer.
  • FIGS. 11A, 11B , 11 C, 11 D, 11 E, 11 F and 11 G show various sample screen images for a Stress Tolerance Test thereof.
  • FIG. 11A shows a picture of an object 70 , which is presented in a single panel on top.
  • One or more exactly matching images are in the set of nine image panels below.
  • panels 71 and 72 match the upper image exactly.
  • the object is to find all matches and click on them in a short period of time.
  • FIGS. 11B, 11C , 11 D, 11 E, 11 F and 11 G show other configurations for matching similar images together.
  • FIG. 12A shows some introductory audio instructions. No stress is provided yet. By the time of the last set of audio instructions, stress inducing commands are introduced, such as reducing the time factor in which to answer a question, and the introduction of verbal chiding, compelling the person being tested to work faster than has been previously done, along with other optional stress-inducing visual or auditory sound effects, such as flashing lights or alarming music.
  • the audio of FIG. 12B illustrates the stress introduced by the auditory portion of the Stress Tolerance Test.
  • FIG. 13A shows a Performance Results screen presentation 80 for the Stress Tolerance Test.
  • FIG. 13A represents the results of the Stress Module in relation to the subject examinee 81 himself or herself and in relation to a general pool of examinees.
  • the wide shaded bar 82 that extends across the entire graph is the measure of the average performance of the general pool of examinees.
  • This Stress Test Module uses sensory indicia to simulate a stressful working environment involving deadlines, accuracy, time urgencies, and negative feedback.
  • the second bar 84 when the graph is viewed from left to right, designated as “Performance Under Stress”, together with the percentile and raw score underneath it, indicates the level of the performance of subject examinee 81 when tested under stressful conditions.
  • the third bar 85 when the graph is viewed from left to right, designated as “Stress Tolerance Index”, together with the percentile and raw score below it, indicates the level of change in the performance of subject examinee 81 between testing with stressful conditions and with non-stressful baseline conditions.
  • the first line of text 86 under the graph indicates the positive performance of the examinee in relation to his own performance.
  • the second line of text 87 under the graph indicates the pattern of error of subject examinee 81 .
  • This error pattern is analyzed and reported as either impulsive or inhibited and then is weighed as “slight”, “moderate” or “significant”.
  • FIG. 13B shows a screen image of an example of the biofeedback monitoring used in conjunction with the Stress Tolerance Test. While FIG. 13B shows that the data is manually inputted, it is known that alternatively the data may be automatically inputted from the sensors and recorded automatically to the system.
  • a “slide show” relaxation phase of non-stressful scenes, such as landscapes, is presented to the subject on the computer screen.
  • FIG. 14 shows a part of a sequence of sample questions with multiple-choice answers from a sample Integrity test.
  • Some questions form related subject matter and these subsets are dispersed.
  • questions 32 and 36 are part of a Drug subset 95 .
  • Questions 35 and 40 are part of a Bribery subset 97 .
  • the Integrity test uses this general principle in sophisticated algorithms comparing responses to similar questions to flag false answers.
  • FIG. 15 shows a screen image for a virtual reality depiction of a work environment.
  • FIG. 15 is one image of a virtual reality simulation of actual outdoor working tasks of a land surveyor.
  • a personal being assessed as a land surveyor views a simulated landscape scene to be measured, as seen through lens 101 of a land surveyor's transiting theodolite sighting telescope.
  • the person being assessed sees a virtual depiction of an actual landscape scene, including man-made structures 102 , such as buildings, natural structures, such as trees and a horizon 104 in the distance.
  • a typical road 105 is depicted in actual perspective, so that the person being assessed can be assessed doing virtual simulations of actual land surveyor's tasks.
  • Persons being assessed can be tested for their endurance under challenging conditions. For example, for a maritime surveyor, swaying of a seagoing boat can be provided in virtual reality images.
  • the present invention utilizes a Computer Optimized Multimedia Intelligence Test (C.O.M.I.T.).
  • C.O.M.I.T. is an individually administered multi-subtest battery based on the Horn-Cattell model of fluid and crystallized intelligence. It yields a comprehensive, valid and reliable composite measure of general intelligence, as well as fluid and crystallized IQ scores.
  • fluid intelligence refers to an individual's ability to devise and apply problem-solving strategies when faced with novel or unusual tasks.
  • Fluid Intelligence reflects the individual's ability to be flexible and apply age-appropriate concept formation, analysis-synthesis and hypothetical-deductive reasoning to solve unfamiliar problems. Fluid intelligence is not heavily influenced by formal school experiences or cultural settings.
  • Crystallized intelligence refers to applying knowledge and skills acquired through formal, school related learning experiences to a current problem. It is heavily influenced by culture, and taps word knowledge, verbal concept formation, categorizations and factual knowledge.
  • the C.O.M.I.T. is an appropriate measure of intelligence for any applied or clinical sheeting requiring assessment of intellectual abilities.
  • the C.O.M.I.T. is flexible and allows for everything from a quick screening to a comprehensive assessment.
  • the C.O.M.I.T. comes in many languages. It has excellent reliability and validity, and correlates significantly with existing measures of intelligence. Since it is both administered and scored by computer, human error and bias introduced by the examiner are eliminated from the testing process.
  • the C.O.M.I.T. test is computer administered, using high-end graphics and professional voice actors.
  • the C.O.M.I.T. yields standard scores (scores with means equal to 100 and standard deviations of 15) for three scales: Composite IQ, Fluid IQ and Crystallized IQ. This permits direct comparisons with global scores earned on numerous measures of achievement such as the Wechsler series of intelligence tests.
  • Sub-tests that measure fluid intelligence are: Visual Analogies, Visual Closure, Visual Memory, and Auditory Memory.
  • Sub-tests that measure crystallized intelligence are: Vocabulary, Information, and Categorization.
  • the five supplemental scales are: Processing Speed, Social Apperception, Auditory Memory with Visual Distraction, Auditory Memory with Auditory Distraction, and Visual Memory with Auditory Distraction.
  • the selection of which sub-tests to administer during a testing session can be completely controlled.
  • the evaluator has the following broad options:
  • the five supplemental sub-tests not included in either the Standard or the Screening battery requires and additional 3-5 minutes each to administer. Testing may be discontinued between sub-tests and continued at a later time.
  • Composite, Fluid and Crystallized IQ scores are presented in colorful graphic displays. Percentile scores are also presented for these composite scores. Graphs displaying performance on individual sub-tests show scaled scores, as well as percentile scores for each sub-test so that subject performance on each individual task can be measured relative to the normative sample. In addition, a special sub-test scatter score uses sophisticated algorithms to calculate sub-test scatter and summarize the individual's relative strengths and weaknesses compared to their own ipsative mean and overall performance. Percentile scores for sub-test performance are also displayed. All sub-test scores (including supplemental sub-test scores) are presented in colorful bar graphs that allow for clear visual comparisons of performance. In addition, supplemental sub-tests involving distraction yield a Distraction Index that indicates the degree to which an individual's performance was affected by distraction, as well as a visual depiction of which distracting modality proved most troublesome for the subject.
  • the present invention also uses a Stress Tolerance Rapid Evaluation Software System (S.T.R.E.S.S.).
  • S.T.R.E.S.S. Stress Tolerance Rapid Evaluation Software System
  • the Stress Tolerance Rapid Evaluation Software System (STRESS) is specifically designed to help psychologists and other professionals rapidly evaluate a subject's ability to remain task focused under stressful conditions. It provides a complete picture of the effects of stress on a subject's ability to sustain attention, maintain attention to detail and stay focused on task completion.
  • the STRESS test has wide applicability across diverse areas of practice, ranging from clinical to applied settings. It is unique and extremely precise because it directly measures an individual's ability to stay task focused under stress, rather than attempting to infer it from what an individual is willing to reveal on a self-assessment questionnaire. Some potential uses for the STRESS test include:
  • STRESS test Most individuals can complete the STRESS test in approximately 15 minutes.
  • the test compares a subject's performance on a visual matching task under two conditions: Non-stress and stress.
  • subjects are presented with up to 4 items.
  • Each item consists of 3D computer-generated shapes: A target shape and nine potential matches.
  • Subjects must identify all the shapes on the screen that are identical to the target. Items occurring in the same order during non-stress and stress conditions were specifically designed to have identical difficulty-levels. Care was taken to reduce ethnic bias and keep the shapes culture-free. Conditions for producing stress were developed in consultation with experts in the field and specifically designed to induce autonomic nervous system arousal and a sense of time urgency. Professional voice actors do all the voice-overs.
  • the STRESS test is administered and scored by computer thus eliminating human error introduced by variations in test administration and scoring.

Abstract

A multi-media method and system performs psychological assessment of an individual for suitability for particular jobs, whereby the individual is shown and provided with discrete visual and auditory stimuli on a multi-media computer screen, so that the individual's responses can be monitored in conjunction with physiological responses received by biofeedback sensors. The computer tallies the test subject's recorded responses. Virtual work environments are optionally displayed, to enable the tested individual to simulate a real work environment. Unlike other job evaluation systems, the system evaluates a job candidate's ability to maintain task focus under pressure. It directly measures performance under pressure, rather than only inferring potential performance based upon selective input from a job candidate.

Description

    RELATED APPLICATIONS
  • This application is a continuation of my pending application Ser. No. 10/101,227 filed Mar. 18, 2002, which application is based upon provisional application No. 60/277,140 filed Mar. 19, 2001, and which application is a continuation-in-part of my application Ser. No. 09/395,548, filed Sep. 11, 1999, now U.S. Pat. No. 6,941,525 dated Dec. 10, 2002, which application is a continuation of application Ser. No. 08/829,488, filed Mar. 27, 1997, now U.S. Pat. No. 6,030,226 dated Feb. 29, 2000, which is based on Provisional Application Ser. No. 60/014,203, filed on Mar. 27, 1996. I claim benefit under 35 USC 120 for these examinable utility applications and claim benefit under 35 USC 119(e) for the aforesaid provisional patent applications.
  • FIELD OF THE INVENTION
  • This invention is related to interactive multi-media applications in psychological testing for vocational assessment, which evaluates a job candidate's ability to maintain task focus under pressure, rather than only inferring performance based upon selective input from a candidate.
  • INCORPORATION BY REFERENCE
  • My above noted prior applications are incorporated by reference herein. My prior patent applications are concerned with applications of multimedia technology for psychological and educational assessment tools. My prior applications are an improvement over conventional psychological and educational assessment tools which have been used for the measurement and evaluation of abilities, aptitudes, personality characteristics and so forth, such as the intelligence quota (IQ) test dating back to 1904 of the French psychologist Alfred Binet that would identify slow learners.
  • The specification and drawings of my patent points out the basic difference between the application of multimedia tools to psychological and educational assessments that are the subject of my invention and, in contrast, the rudimentary skill acquisition or remedial training tools commonly known as computer-assisted instruction (CAI). In computer assisted instruction (CAI), courseware is provided for teaching presentations in various learning areas. Unlike my prior invention, in computer assisted instruction (CAI), the subjective factors of reliability, validity, bias, and standardization are seldom of central importance because of the objective teaching nature of the CAI courseware.
  • In contrast, my patent discloses that these subjective factors are of primary importance in the psychological and educational assessment tools, and that accurate determination of the tested subject's basals and ceilings is required for the proper administration and scoring of the psychological assessment test, which is used in education. My patent also points out that video and audio presentations are mainly matters of variable style or perceived effectiveness in the computer assisted instruction (CAI) environment.
  • Further in contrast, with respect to psychological assessment, assessment tools are rigorously field tested (i.e. “normed”) for reliability and validity, and their mode of administration is precisely stipulated, and any deviation may violate the conditions under which it had been normed.
  • My patent points out that psychological and educational assessment tools can utilize the full range of multi-media presentation features which closely simulates the manual version of a normed test. In my patented application of multimedia technology, the standards used to create the manual test are easily translated to the computer test by a comparative study of the similarity of test scores of computer assisted versus manual testing.
  • My patent also emphasizes that Computer Assisted Instruction (CAI) is used to teach new material, to review learned material as in drill and practice, to teach productivity techniques such as word processing and to teach programming, and to keep track of scores or to adjust the level of difficulty.
  • In contrast, the Computer Assisted Assessment (CAA) of my invention is an assessment tool, so that teaching is not the objective. Scoring of Computer Assisted Assessment (CAA) is sophisticated with raw scores, standard scores, sub-tests, age equivalents, and basals and ceilings. The testing rules of Computer Assisted Assessment (CAA) are rigid with no deviation allowed.
  • My prior patent emphasizes that a clinical psychological tester can use the Computer Assisted Assessment (CAA) with its attendant multimedia tools, in the area of question sequencing and scoring related to establishing the critical range of the tested subject, and that the subject's basals and ceilings are determined by runs of right or wrong answers within a particular sequence. My patent also discusses a method for reducing bias, such as social class prejudice or the assumption that shy or disruptive children are learning impaired.
  • My patent also overcomes that fact that some clinicians may be very proficient in their respective fields, but that they may not speak clearly, which causes confusion. My patent reduces the probability that a tested subject will get a question wrong just because he or she doesn't understand the examiner, especially in English as a Second Language (ESL) subjects.
  • My patent also overcomes the limitations of standard Computer software for test scoring and analysis of error patterns, which is available for manually administered tests. But these computer-assisted tests are not constantly re-evaluated, and the computer provides no computer guidance in administrating the test, since this is done as an ad-hoc procedure after the test is over.
  • In the “Summary of the Invention” portion of my patent, there is described a method and system to perform psychological and educational assessment of subjects using multi-media technology.
  • The method described in my patent is for psychological and educational evaluation of human subjects, which includes the steps of providing a human subject, such as a student, to be evaluated, with a multi-media capable computer having a view monitor means, such as a VDT terminal, and a computer input-response means, such as a keyboard, a mouse or a touch screen. In that patent, it is noted that a human psychological tester optionally operates the multi-media computer and conducts the psychological evaluation of the student.
  • My patent also discusses the option that the multi-media computer has a linguistic adaptation means, such as a sound card and data associated therewith, which can produce sounds, such as test instructions, in the most appropriate language and dialect of the test taker, such as a student, so that the human tester can provide the human subject with instructions for responding to the psychological evaluation.
  • As for multi-media technology, my patent discusses that a human subject being psychologically evaluated is exposed to computer-generated multi-media psychological evaluative probing, so that the human subject can respond to the computer-generated psychological evaluative probing.
  • This feature is reiterated in my new patent application for my present invention in the field of psychological evaluation of subjects during psychological vocational assessment. In both my patent and the present invention herein, the computer tallies and records the subject's responses and analyzes the human subject's recorded responses. Afterward the computer automatically prepares a report based upon the human subject's recorded responses, and transfers its report, such as by transfer to a computer-readable diskette or by printing to a printer. In both my earlier patent and in my present invention for vocational assessment, the psychological evaluation of the human subject is provided by at least one software program stored in a randomly accessible non-volatile computer memory, such as the computer's hard drive, for providing at least one type of psychological evaluation. The at least one software program further includes a plurality of discrete audible and/or visual segments such as questions to be responded to by the human subject being evaluated.
  • The discrete audible segments include audibly reproducible signals stored in a randomly accessible non-volatile computer memory for producing sounds to be responded to by the human subject being evaluated, such as spoken phrases to be responded to by the human subject. The spoken phrases are selected from a plurality of user-selectable languages and/or dialects.
  • The discrete visual segments include visually-reproducible signals stored in a randomly accessible non-volatile computer memory for producing images to be responded to by the human subject being evaluated, such as text selected from a plurality of psychological evaluation question items stored as text written in a plurality of languages and/or dialects and/or character sets, such as Chinese or Cyrillic, among others.
  • My patent also discusses the fact that to assist in evaluating temporal time oriented tasks, such as allowing a person decide which of a plurality of moving objects, such as balls, moves fastest, or the calculated destination thereof, optionally the discrete visual segments may include computer-generated motion for psycho-motor evaluation of human subjects.
  • My patent discusses that computer-generated motion may include a plurality of user selectable visually perceived motions, such as the moving balls, at differing speeds and/or velocities and/or accelerations.
  • Moreover, my patent also discusses that the software program can present audible psychological probing tasks in a variety of user-selected languages and dialects, wherein the languages and dialects are respectively stored discretely in randomly accessible computer non-volatile memory, such as the computer's hard drive or upon a removable diskette. In both my patent and in my present invention for vocational assessment, the method of using multi-media technology in psychological evaluations may include an evaluation of the human subject's emotional status, mental health, learning ability, neurological impairment status, educational status, educational achievement status, aptitude for future education or aptitude for future vocational education.
  • In the “Brief Description of the Drawings” portion of my patent, there was disclosed multi-media equipment which is used in a test session for psychological evaluation.
  • My patent disclosed typical video display screen's views of various multi-picture panels used in the computer assisted psychological assessment.
  • My drawings also disclosed video displays using screen panels, optional animation and voice response modes.
  • The drawings of my patent also show a video display screen sequence of the multi-media system with clarifying instructions and “Dynamic Linguistic Adaptation” portions.
  • My patent also shows various exemplary video display screens that illustrate modeling proper response sequences for sample questions.
  • My patent also provides for an input response means 23 which is used to input communicated sensory data from the human subject to multi-media computer to conduct a psychological evaluation of the human subject.
  • My patent also discloses how a computer-responsive user selectable means, such as a keyboard, allows a psychological tester to select pre-recorded test instructions in appropriate languages and dialects appropriate to the language and dialect of human subject being tested.
  • My patent also uses the multi-media tool of an audible playback which audibly communicates one or more sets of pre-recorded test instructions, with audible playbacks having sound cards and data associated therewith, to produce audible test instruction sounds, in the appropriate language and dialect thereof.
  • My patent also discloses a multi-media software program that includes a plurality of discrete audible and/or visual segments such as questions to be responded to by the human subject being evaluated. In my patent, a user-selectable set of computer-generated multi-media psychological evaluative probing data communicates with the audible playback, and a recorder records responses of the human subject in response to the computer-generated psychological evaluative probing. The human subject's responses are tallied.
  • In my patent, a computer output printer automatically prepares a written record of the human subject's recorded responses in a written report, which report is transferred to a computer, and is stored in the randomly accessible non-volatile computer memory of the multi-media computer.
  • My patent also shows a psychological assessment session, using a multi-media system to give the same type of test as is done in conventional human-to-human evaluations. My patent discloses how a tested human subject sits facing a computer screen on a video display terminal of a multi-media personal computer, wherein the subject uses pointing device to indicate his or her answers.
  • My patent also discusses various multi-media computer components, such as a mother board containing a microprocessor and other modules and support electronics, with optional extra random access memory (RAM), a hard drive, a floppy drive using removable media (diskettes), a CD-ROM with a removable compact disk (CD), a sound board and a variety of sockets or ports to support a speaker, a microphone, a display, a mouse port and a keyboard socket.
  • My patent gives a non-limiting example of an appropriate multi-media computer, such as a Packard Bell model AX3510, which has 8 MB RAM, a 1000 MB hard drive, Quad speed CD-ROM, stereo speakers and an Intel 100 MHZ Pentium processor.
  • My patent explains that Computer Assisted Assessment tools may run on a 33 MHZ '386 class processor with a 100 MB hard drive, single speed CD-ROM and 4 MB of RAM, or a notebook computer having similar capability with integral liquid crystal display (LCD).
  • My patent discloses that any type of pointing device can be used; such as, for example, a mouse, track ball, video game control, joy stick, or even a finger when used with a “touch screen”, as well as other display technologies such as plasma displays or Texas Instrument's micro-mirrors, which can support 3-D presentation displays, which are most appropriate in psychological assessment tests.
  • My patent shows that educational and psychological assessment test content is prepared using state-of-the-art multi-media authoring tools combining audio, picture and text sources, which improves over the manual administration of these psychological assessment tools, so that the automatic nature of using a computer precludes bias in test administration.
  • My patent reiterates that computer administration enhances on-task performance of the person being tested, in a non-biased fashion, so that test standardization is assured by computer administration.
  • My patent shows that multi-media technology can selectively first show an image with a plurality of objects, and then can eliminate a number of the presented objects to focus the tested subject on an appropriate response.
  • My patent also shows that animation can be used in creating new measures dealing with temporal issues, such as movement of objects, which is also appropriate in my present invention which, among other features, can virtually simulate a vocational environment of a job candidate. My present invention also shows that animation can also test cognitive skills in predicting events related to the virtual work environment. In my present invention, a multi-media computer is ideally suited for assessment tools that require the use of timing, to replace assessment evaluations requiring the use of a stopwatch by the clinician to either time the presentation of materials or the period for timed responses (or both).
  • My patent also shows that a multi-media computer may have voice recognition software of proper quality that it can reliably recognize the utterances of a tested subject and that a microphone can be used to field the responses and determine if an answer is right or wrong, or if more than one response has been given.
  • Such state-of-the-art of voice recognition software is disclosed in software of Syracuse Language Centers, which run on a 386-class computer with 4MB of RAM and are capable of analyzing the pronunciation of words automatically.
  • My patent also discloses that different keys on a keyboard or pointer can also be linked to specific verbal responses or prompts. For example, if the test subject gives an ambiguous answer, the computer can audibly say, “Explain what you mean.” In the present invention, the computer can evaluate stressful assessment situations, by uttering critical statements, such as, “You're not working fast enough,” and then can use biofeedback monitoring to evaluate the subject under stress.
  • My patent discusses the use of appropriate language dialects, as well as an “assisted voice recognition” system which uses a keyboard and no microphone. In this system, keys on the keyboard are mapped onto special functions by the testing software, wherein responses of a tested subject may be responded to with an audio feedback such as, “Please respond with only one answer” every time one of the “Invalid” keys is pushed.
  • My patent also discusses how incorrect answers can be responded to if answered incorrectly. The test can be automatically terminated if the test subject never gets beyond an initial stage. After the test, a raw score is updated, the basal/ceiling analysis is performed, the test score is updated, the sequence is changed if necessary, and the subsequent following question is presented (or the test is ended). This general flow can be used with pointing as well as verbal answers. It is understood that other logical flows can be used to achieve an equivalent result without deviating from the scope of this invention.
  • My patent also demonstrates various video display screens displaying multi-media techniques in modeling appropriate responses for sample questions. The drawing Figures show screens with pictures of objects wherein the tested subject is asked to point to one of the objects and if the answer is incorrect, the respective display screen shows the image corresponding to the correct answer. By not showing the other two images in the same screen in the present invention, the student's attention is focused on the correct answer.
  • My patent also shows a typical example known as a CREVT-CA program, which shows how the method and system of the present invention is used in one illustrative example. In this example, a sample psychological/educational assessment test is run in a computer program designated as the “Comprehensive Responsive and Expressive Vocabulary Test-Computer Administered” (CREVT-CA).
  • To start the CREVT-CA program, the user double clicks on the CREVT-CA icon in the CREVT-CA Program Group. If the user has installed the program in WINDOWS 95®, from the start menu, the user selects “Programs, CREVT Program Group”, with the CREVT-CA icon. After the opening title, the Main Menu appears. Six choices on this screen are available to the user, such as:
  • (OK) Administer Tests
  • (OK) Score Expressive Subtest
  • (OK) View a Previous Record
  • (OK) Change the Password
  • (OK) Recharging Information
  • (OK) Exit
  • To choose an option, the user points and clicks on the box next to the operation the user wishes to bring forth.
  • My patent also shows how a typical program, such as a CREVT-CA is used to administer a test to an individual.
  • In the CREVT-CA example, the user can choose to administer both the Receptive and Expressive sub-tests or individual sub-tests.
  • Then, user the completes the demographic information screen.
  • The user selects an appropriate language and dialect of the test subject, such as a child, such as “French-Quebecois” vs. “French-Haitian Creole”, or “English-Australian” vs. “English-Southern American” and the like. When this is done, the user clicks on the “OK” button to continue.
  • Finally, the user chooses which form of the test the user would like to administer—Form A or Form B. If the user is administering both sub-tests, the form selection applies to both sub-tests.
  • Whether the user has chosen to administer the Receptive Vocabulary Subtest, or both Sub-tests, the testing begins with the Receptive subtest. The subject is seated in front of the computer with the mouse in hand (in other test environments, the examiner may input information from the subject, who is tested in a hands-free environment). During the Receptive portion of the test, a voice in a pre-selected language and dialect pronounces a word while the subject views the pictures. In this example, the subject simply uses a pointer, such as a computer mouse to point and click on a picture to indicate an answer.
  • As soon as a ceiling is reached (when the subject misses two consecutive items in the set of items) the program automatically moves on to the next set of pictures.
  • If the user is using the CREVT-CA program to administer the Receptive portion of the test but is administering the Expressive portion without the computer's assistance, the user can still have the CREVT-CA compute the standard scores, percentiles, and age equivalents for both sub-tests together. When the Expressive and Receptive sub-tests are completed, and after the user has finished the Receptive test on the computer, the user simply clicks on the “yes” button to choose to score the Expressive subtest as well. The CREVT-CA program asks the user to enter the raw score of the Expressive subtest and then computes the Receptive, Expressive, and Composite scores.
  • After the Receptive subtest is completed, the user needs to enter the user's password to be able to view the scores (the password is initially set to “CREVT”). First, the screen displays a graph of the percentage of correct answers per set of pictures. This graph can be printed by clicking on the “OK” button next to the word “Print” at the bottom of the screen. To continue on and to view the subject's scores, the user clicks on the “OK” button next to the words “Continue” and “View Test Scores”.
  • The Expressive portion of the test is given on the computer after the Receptive portion or by itself. The computer gives the instructions orally and asks the subject to define a series of words.
  • The Expressive Subtest automatically ends when either a ceiling is reached (i.e., when the subject misses three consecutive items) or when the last (25th) item is administered.
  • According to my patent, when the testing is finished for a particular subject, the examiner enters a password (the password is initially set to “CREVT”) to view a comprehensive report of the test results on the computer screen. The examiner can then print out a hard copy of the report by clicking on the “PRINT” button at the bottom of the scores screen. If the Expressive subtest was administered and some information was saved in the “SUBJECT'S REPLY” text boxes, the examiner has the option of also printing out those replies.
  • The CREVT-CA program also allows the user to graph the standard scores obtained by the subject, along with up to seven standard scores with means of 100 and standard deviations of 15 obtained when using other tests with the subject. To print, the user clicks on the “OK” button next to the work “PRINT” on the bottom of screen displaying the graph.
  • To view the scores of a subject who took the test previously, the user chooses to view a previous record option from the Main Menu. The user enters the name of the subject whose record the user wants to see. The user can search for a record by last name only or by first and last names. The user can also choose to view the database record by record. Once the desired record is displayed on the screen, the user can choose to view either the subject's scores or the subject's responses to the Expressive Vocabulary subtest. The user can also print out hard copies of either of these items.
  • In my present invention, it is noted that many of the aforesaid multi-media applications can be specifically used in the field of vocational psychological assessment, to ascertain whether a tested subject is suitably fitted for a particular job. Not only can the person's cognitive aptitude be determined, but the tested person can be monitored while subjected to non-stressful and stressful situations, to simulate the rigors of a particular job or the person's suitability in general for a particular job.
  • BACKGROUND OF THE INVENTION
  • Various attempts have been made to use computers in vocational assessment. For example, U.S. Pat. No. 5,326,270 of Ostby discloses use of a computer for pre-employment screening, but it is more of a role playing game that is also applicable to training. The emphasis on “path analysis” in assessing task-processing style has no counterpart in the present invention.
  • U.S. Pat. No. 4,627,818 of Von Fellenberg discloses a very exacting method for presentation of paired questions and analysis of their results for identifying subject's psychological traits.
  • U.S. Pat. No. 4,683,891 of Cornellier discloses a computerized employment training using biofeedback monitoring to enable a subject to increase his tolerance of stress and enable him to perform tasks under more stressful conditions, using goal oriented cognitive tasks.
  • U.S. Pat. No. 5,551,880 of Bonnstetter discloses an Employee Success Prediction System, using a computer to administer testing, but appears not to use graphics or multi-media. The testing in Bonnstetter '880 uses questionnaires. Both behavioral and value profiles of a subject are obtained.
  • In addition, it has been found previously in the prior art, primarily from testing for integrity on polygraph tests, that questions that take longer to answer and that exhibit more response time latency variability are not being answered truthfully.
  • However, these prior art patents and polygraph integrity testing do not disclose a multi-media method and system which performs psychological assessment of an individual for suitability for particular jobs, whereby the individual is shown and provided with discrete visual and auditory stimuli on a multi-media computer screen, so that the individual's responses can be monitored in conjunction with physiological responses received by biofeedback sensors.
  • These prior art patents and integrity testing also do not disclose display of virtual work environments, to enable the tested individual to simulate a real work environment, so that a job candidate's ability to maintain task focus under pressure can be measured, rather than only inferring potential performance based upon selective input from a job candidate.
  • OBJECTS OF THE INVENTION
  • It is therefore an object of the present invention to provide a computer administered modular multi-media normed personnel evaluation and vocational assessment system.
  • It is also an object to be able to evaluate test performance, interpretation and scoring criteria which evolves over time to reflect changes in the population subjected to the various test modules to establish norms more representative of the current population.
  • It is yet another object to provide multi-media software and tangible technology which is delivered as portable media, such as CD-ROM, diskette, DVD, computer compatible input media, or software which is downloaded and administered from the Internet.
  • It is also an object to provide for pre-employment screening, on the job promotion/retention/length of development screening and pre and post training evaluations which contain job performance modules as well as aptitude subtest modules.
  • It is also an object to provide a menu derived from subject population statistics for particular job titles, which is presented for reflecting the suitability of various test modules, the suggested weight for each module and the pass/fail limits of each module compared to subject population statistics.
  • It is also an object to provide a vocational assessment tool wherein the test administrator can modify the module selection, the pass/fail limits, and the weights assigned to each module for a particular job title or for a particular individual, in a variety of job performance modules.
  • It is also an object to provide a computer administered multi-media stress tolerance test that compares a subject's performance on a visual matching task under two conditions, non-stress and stress, by using test shapes which are selected to minimize ethnic bias and cultural differences.
  • It is further yet another object to provide a multi-media vocational assessment system which utilizes physiological sensors to actually measure and quantify changes in the autonomic nervous system of the person being tested, thereby detecting the onset of such changes and the degree of such changes during the stress test.
  • It is also an object to provide a vocational system which can test a subject's reaction to psychologically induced stress, with that induced by physical means, such as by a standard treadmill test sequence.
  • It is also an object to provide a multi-media vocational assessment, whereby images are revealed progressively, wherein the subject identifies emerging pictures and the required by the subject to make his or her identification is evaluated.
  • It is yet a further object to provide a computer administered integrity test module wherein questions are dispersed and repeated throughout a test, to detect deceptive answers.
  • It is also an object to provide a computer-generated vocational assessment test which uses virtual reality images and sounds to simulate various vocational work environments, to ascertain the best placement of a potential worker.
  • It is therefore also an object to provide a computer-generated vocational assessment system which uses virtual reality images to selectively reject persons who have vestibular or motor coordination problems not suited for the work environment being simulated.
  • It is also an object of the present invention to improve over the disadvantages of the prior art.
  • SUMMARY OF THE INVENTION
  • In keeping with these objects and others which may become apparent, the present invention provides a battery of tests which are administered to an individual by a computer using multi-media features. This normed testing can be used to assess the suitability of an applicant for employment, although other versions of the test emphasizing the stress tolerance module can also be used for screening candidates for high-stress or high-risk mission critical assignments. The primary objective of this invention is to reliably and accurately predict real world success using an easily administered rapid assessment technique.
  • As the performance and scoring of these normed tests evolve over time, an important feature of this assessment system is to maintain a database of field experience to continually refine the norms to insure that they are representative of the current population of individuals subjected to these tests. Updates of these norms are distributed to customers. The actual software product is delivered to customers as portable media such as CD-ROM, diskettes, DVD or similar computer input media. It is also available for download to a test administrator in secure fashion from an Internet web site. Direct use from a web site on the Internet or on a company intranet is another option for test delivery.
  • For example, the multi-media software may have an interface utilizing one or more computer input hardware devices such as a mouse, a keyboard, a touch pad, a microphone, a camera, physiological sensors or any other computer input device. The software is delivered to a test administrator as portable media such as CD-ROM, diskette, ROM, non-volatile RAM, flash media, DVD or other computer compatible input media.
  • The software may be also be downloaded from the Internet, where access is obtained through the Internet. The tests may be administered through the Internet and the related administrative functions may be performed through the Internet. When the Internet is used, there may be secured communication, wherein the secured communication is implemented with one or more security technologies, such as an internet protocol security, a secure socket layer, a private key encryption and/or a public key encryption or may utilize a VPN or other form of secure tunneling technology.
  • Moreover, the present invention includes a personnel evaluation system with one or more multimedia computers for sending and receiving audio and visual stimuli, each connected to respective central processing units to administer a pre-employment evaluation, a pre-promotion evaluation, a pre or post training evaluation, a personnel development/leadership training evaluation, or other form of employment evaluation application evaluation and to perform an analysis of input data. The software application stores and evaluates all input data and includes one or more modules to monitor and re-scale non-truthful responses of prospective employees, wherein one or more interfaces constantly monitor a physiological state of a subject. These interfaces induce stress inducing audio and visual stimuli and administer the evaluations with race, gender, ethnic, and cultural neutral stimuli. Optionally the interface may interact with the subject in their native language, and dialect. The interface assesses various aspects of the subject which might affect work performance, such as intelligence, interest, values, emotional status, mental health, neurological impairment, education, social apperception and aptitude of the subject being tested. The system can detect for untruthful responses and re-scale values of questions depending on the type of test and detect physiological changes in the subject. These physiological factors which are monitored by biometric sensors include measuring one or more of the following: galvanic skin responses, muscle tone, facial expressions, pupil dilation, heart rate, EKG electrocardiogram responses, blood pressure, thoracic respiration, abdominal respiration, saliva composition.
  • One popular use for this invention is for pre-employment screening. A menu derived from subject population statistics is presented to the test administrator by the software which displays the suitability of the various test modules, the pass/fail limits, and the weight assigned to each module, all with respect to a particular job title. The administrator can modify these suggested parameters to tailor the screening to his department needs. The job performance modules include tests for Integrity, Aptitude, Loyalty, Stress Tolerance, Memory, Social Awareness, Communications Skills, Performance Efficiency, Distractibility, and Impulsivity. Aptitude sub-tests include Visual Closure, Visual Analogies, Categorization, Information, Personality, Interests, and Values.
  • The computer administered multi-media stress tolerance test module compares a subject's performance on a visual matching task under two conditions, non-stress and stress. The visual matching task uses pictures of 3-D shapes and objects that are carefully selected to avoid bias to different ethnic groups or socio-economic status. Stress factors used are audio coaching telling the user to hurry, chiding the user for making errors, and short time limits for test segments. For certain applications such as screening candidates for critical missions, physiological sensors that actually measure and quantify changes in the autonomic nervous system are attached to the subject. These sensors detect the onset and degree of such changes before, during and after the stress test. Sensors on the subject's chest, arm, and fingers measure heart rate, EKG electrocardiogram responses, skin temperature, skin conductance, muscle tension, respiration rate, facial expressions, eye pupil dilation, galvanic skin responses, muscle tone, facial expressions, thoracic respiration, abdominal respiration, saliva composition and blood pressure and the changes and degree of changes, in all of the above, before, during and after the stress test. Additionally, digital recording of facial expressions and pupil dilation are made. Thus, these changes and the degree of change of these physiological parameters are monitored during the entire test module session. Combined with subject recovery time data, a very comprehensive study of the reaction to physiologically induced stress can be made in a clinical environment. Examination of difference scores for correct matches, errors of commission, and errors of omission provides a rapid and precise indication as to the degree to which an individual is affected by stress as well as the analysis of the likely type of error an individual would make under stress.
  • Several aptitude sub-tests using computer multi-media techniques are used. Although they are similar to namesake paper-and-pencil tests in objective, they are greatly enhanced by the capability of the computer. Professional voice actors are used to present audio instructions or test questions thus eliminating variability due to communications capabilities of “test givers”.
  • For example, modules to test memory and distractibility is very difficult to administer in a consistent fashion in any other manner.
  • All of the following combinations are administered: visual memory, auditory memory, visual memory with visual distraction, visual memory with auditory distraction, auditory memory with visual distraction, and auditory memory with auditory distraction. The subject's performance is also measured on memory modules without distracters, to measure the degree of subject distractibility and to measure the specific modality of subject results, which compares performance without and with distractions, to measure the degree and type of specific modality of distraction that the subject finds most disturbing.
  • A computer administered multi-media visual analogies test module uses a combination of audio instruction with presentation of two 3-D color pictures of geometric shapes related in some way followed by a third picture which is related in the same way to one of five choices presented.
  • The shapes are chosen to minimize bias to different ethnic groups or socio-economic status. A visual closure test is efficiently administered as a picture is revealed progressively as it replaces random background elements.
  • The test is scored both by the accuracy with which the subject identifies the emerging picture as well as by the degree of completion (i.e. time) required by the subject to make the identification. A computer administered social apperception test presents an array of high-resolution pictures of people in a variety of postures and facial expressions. The audio presents a spoken statement. The subject is to choose the picture of the person most likely to have made that statement using visual body language and expression cues to pick up the subtle nuances of emotion depicted which reflect that of the spoken statement, as well as the nuances of expression and emotion involved.
  • A computer administered integrity test involves a set of written questions with multiple-choice answers. Subsets of questions with related subject matter are dispersed throughout. The answers to the related questions are analyzed as to response time latency variability to detect attempted deception.
  • A computer-generated and administered vocational assessment system uses virtual reality techniques to simulate a variety of vocational work environments or specific situations to ascertain the best placement of a potential employee. This is accomplished by generating specialized test modules or by substituting virtual reality images and sounds within the format of the job performance and aptitude sub-tests described above.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The present invention can best be understood in conjunction with the drawings of my aforementioned U.S. Pat. No. 6,030,226, as well as the following drawings in which:
  • FIG. 1 is a test module menu for a “driver” job title of the multi-media vocational assessment system of the present invention;
  • FIG. 2 is a screen image and audio for a Vocabulary test thereof;
  • FIG. 3 is a screen image and audio for an Information test thereof;
  • FIG. 4 is a screen image and audio for a Categorization test thereof;
  • FIG. 5 is a screen image and audio for a Social Apperception test thereof;
  • FIG. 6 is a screen image and audio for an Auditory Memory test thereof.
  • FIG. 7 is a sample for a screen image and audio for an Auditory Memory with a Visual Distraction test thereof.
  • FIG. 8 shows audio instructions for a Visual Memory test thereof;
  • FIG. 9 shows a screen image and an audio for a Visual Memory test thereof;
  • FIG. 10 shows a screen image and an audio for a Visual Closure test thereof;
  • FIGS. 11A, 11B, 11C, 11D, 11E, 11F and 11G show various sample screen images for a Stress Tolerance Test thereof;
  • FIG. 12A displays audio for a Stress Tolerance test thereof;
  • FIG. 12B displays a continuation of the audio for Stress Tolerance Test as in FIG. 12A;
  • FIG. 13A shows a performance results screen for a Stress Tolerance Test thereof;
  • FIG. 13B shows a biofeedback monitoring screen for a Stress Tolerance Test thereof;
  • FIGS. 13C, 13D and 13E show various relaxation phase screens following a Stress Tolerance Test thereof;
  • FIG. 14 shows a screen image from a sample for an Integrity test thereof; and,
  • FIG. 15 shows a screen image for a virtual reality depiction of a work environment.
  • DETAILED DESCRIPTION OF THE INVENTION
  • When used for pre-employment screening of individuals, the testing system of this invention presents an initial menu screen 1 to the test administrator as shown in FIG. 1. The name “SAVVY RECRUITER” indicates a trade name of Applicant. Menu screen 1 is identified with a specific job title 2, which is a “general manager” for this example. The job attributes 3 are presented in a column format to the left and their applicability to this specific job title 2 is presented in “weight” 4 column to the immediate right. A job attribute 3 such as “Aptitude” is weighted as being more relevant than “Memory” for the job title 2 of general manager.
  • For a specific job attribute 3, such as “Aptitude”, a report may be generated by clicking on an option 5 for a report and option 6 for display of relevant test weight. In addition, scoring threshold 7 may be displayed to indicate the desirability of the applicant for a particular position such as job title 2. Time column 8 presents an estimate of the time elapsed. The bottom right portion of menu screen 1 presents similar entries for Aptitude Sub-tests 9. The test administrator can alter these recommended settings to tailor the test evaluations for the company's specific requirements.
  • FIG. 2 shows an introductory sample screen for a Vocabulary test. A professional voice actor speaks the directions 13 printed below. The subject is prompted with an auditory cue 14 such as “chair”, upon which time he or she should click on the chair 15 picture. If he or she realizes that an error has been made, the repeat symbol 12 can be clicked to get a second try. Continuing with the audio 14, a sharpener 16 and a calculator 17 should be indicated.
  • A similarly administered test, “Information,” is shown in the screen image of FIG. 3, wherein Audio instructions 21 are presented. Then audio statements 22 are presented and the tested subject is to indicate the image of the item which answers the question such as eye 24, nose 23 or teeth 25. Button 20 can be clicked to correct an answer.
  • FIG. 4 shows a screen image of a Categorization test. As per audio instructions 27, the objective is to select the picture of the item which is not like the others. Audio statements 28 would only be given during sample questions. In this case, the picture of the lion 29 is the correct answer.
  • FIG. 5 is a screen image of a sample question from a Social Apperception test. Audio instructions 33 define the objective as to “choose the person that goes with what you heard”. Button image 32 can be clicked to change an answer. Sample statement 34 goes with image 35 since the statement displays anger.
  • FIG. 6 shows a screen from an Auditory Memory test. Audio directions 38 and examples 39 are strictly an introduction. The actual test presents a stream of numbers using audio. The subject has to enter the digits in the same order by clicking on number symbols 40. He or she may click cancel button 41 to try again, or signal OK 42 if he or she feels his or her answer was the correct sequence.
  • FIG. 7 is a sample introductory screen from an Auditory Memory with Visual Distractions. The audio instructions, 45 and 46, explain the added visual “clutter” 43 consisting of 3-D images of spheres surrounding the numeric display 40. The test begins as the sequences 47 are read on the audio stream.
  • Visual Memory test audio directions 50 and 51 are shown in FIG. 8.
  • The corresponding screen image is shown in FIG. 9. Shapes are presented in a sequence one at a time in frame 52. The subject has to click on the shape images continuously displayed in the same order and then click OK 56 when done. The subject can change his answer once by first pressing CANCEL 55. In the case shown, the image shown in frame 52 is the same shape as item 53 below. Audio instructions 54 comment on the answer of the subject, and give instructions for further proceedings.
  • FIG. 10 is a screen image of a Visual Closure test. Basic audio instruction 60 is presented for this sample. Explanation 61 is also for illustration. The object image 63 emerges as more of random background 62 is converted to image 63 in a progressive fashion. When the subject recognizes the picture as one of the four choices 64 presented, he or she clicks on the matching image. This is a very smooth operation, and the time to answer is accurately noted by the computer.
  • FIGS. 11A, 11B, 11C, 11D, 11E, 11F and 11G show various sample screen images for a Stress Tolerance Test thereof. For example, FIG. 11A shows a picture of an object 70, which is presented in a single panel on top. One or more exactly matching images are in the set of nine image panels below. In this example of FIG. 11A, panels 71 and 72 match the upper image exactly. The object is to find all matches and click on them in a short period of time. FIGS. 11B, 11C, 11D, 11E, 11F and 11G show other configurations for matching similar images together.
  • FIG. 12A shows some introductory audio instructions. No stress is provided yet. By the time of the last set of audio instructions, stress inducing commands are introduced, such as reducing the time factor in which to answer a question, and the introduction of verbal chiding, compelling the person being tested to work faster than has been previously done, along with other optional stress-inducing visual or auditory sound effects, such as flashing lights or alarming music.
  • The audio of FIG. 12B illustrates the stress introduced by the auditory portion of the Stress Tolerance Test.
  • FIG. 13A shows a Performance Results screen presentation 80 for the Stress Tolerance Test. For example, FIG. 13A represents the results of the Stress Module in relation to the subject examinee 81 himself or herself and in relation to a general pool of examinees. The wide shaded bar 82 that extends across the entire graph is the measure of the average performance of the general pool of examinees. This Stress Test Module uses sensory indicia to simulate a stressful working environment involving deadlines, accuracy, time urgencies, and negative feedback. The first bar 83 on the left designated as “Baseline Performance”, together with the percentile and raw score below it, indicates the level of the performance of subject examinee 81 when tested in a non-stressful environment, calm environment.
  • The second bar 84, when the graph is viewed from left to right, designated as “Performance Under Stress”, together with the percentile and raw score underneath it, indicates the level of the performance of subject examinee 81 when tested under stressful conditions.
  • The third bar 85, when the graph is viewed from left to right, designated as “Stress Tolerance Index”, together with the percentile and raw score below it, indicates the level of change in the performance of subject examinee 81 between testing with stressful conditions and with non-stressful baseline conditions.
  • The first line of text 86 under the graph indicates the positive performance of the examinee in relation to his own performance.
  • The second line of text 87 under the graph indicates the pattern of error of subject examinee 81. This error pattern is analyzed and reported as either impulsive or inhibited and then is weighed as “slight”, “moderate” or “significant”.
  • FIG. 13B shows a screen image of an example of the biofeedback monitoring used in conjunction with the Stress Tolerance Test. While FIG. 13B shows that the data is manually inputted, it is known that alternatively the data may be automatically inputted from the sensors and recorded automatically to the system.
  • During this time, as shown in FIGS. 13C, 13D and 13E, a “slide show” relaxation phase of non-stressful scenes, such as landscapes, is presented to the subject on the computer screen.
  • FIG. 14 shows a part of a sequence of sample questions with multiple-choice answers from a sample Integrity test. Some questions form related subject matter and these subsets are dispersed. For example, questions 32 and 36 are part of a Drug subset 95. Questions 35 and 40 are part of a Bribery subset 97. As noted above in the “Background of the Invention” portion, it has been found previously in the prior art, primarily from testing on polygraph tests, that questions that take longer to answer and that exhibit more response time latency variability are not being answered truthfully. The Integrity test uses this general principle in sophisticated algorithms comparing responses to similar questions to flag false answers.
  • FIG. 15 shows a screen image for a virtual reality depiction of a work environment. For example, FIG. 15 is one image of a virtual reality simulation of actual outdoor working tasks of a land surveyor. In FIG. 15, a personal being assessed as a land surveyor views a simulated landscape scene to be measured, as seen through lens 101 of a land surveyor's transiting theodolite sighting telescope. The person being assessed sees a virtual depiction of an actual landscape scene, including man-made structures 102, such as buildings, natural structures, such as trees and a horizon 104 in the distance. A typical road 105 is depicted in actual perspective, so that the person being assessed can be assessed doing virtual simulations of actual land surveyor's tasks. Persons being assessed can be tested for their endurance under challenging conditions. For example, for a maritime surveyor, swaying of a seagoing boat can be provided in virtual reality images.
  • OPERATION OF THE INVENTION
  • The present invention utilizes a Computer Optimized Multimedia Intelligence Test (C.O.M.I.T.). The C.O.M.I.T. is an individually administered multi-subtest battery based on the Horn-Cattell model of fluid and crystallized intelligence. It yields a comprehensive, valid and reliable composite measure of general intelligence, as well as fluid and crystallized IQ scores. In the Horn-Cattell model, fluid intelligence refers to an individual's ability to devise and apply problem-solving strategies when faced with novel or unusual tasks. Fluid Intelligence reflects the individual's ability to be flexible and apply age-appropriate concept formation, analysis-synthesis and hypothetical-deductive reasoning to solve unfamiliar problems. Fluid intelligence is not heavily influenced by formal school experiences or cultural settings.
  • Crystallized intelligence, on the other hand, refers to applying knowledge and skills acquired through formal, school related learning experiences to a current problem. It is heavily influenced by culture, and taps word knowledge, verbal concept formation, categorizations and factual knowledge.
  • The C.O.M.I.T. is an appropriate measure of intelligence for any applied or clinical sheeting requiring assessment of intellectual abilities. The C.O.M.I.T. is flexible and allows for everything from a quick screening to a comprehensive assessment. The C.O.M.I.T. comes in many languages. It has excellent reliability and validity, and correlates significantly with existing measures of intelligence. Since it is both administered and scored by computer, human error and bias introduced by the examiner are eliminated from the testing process.
  • The C.O.M.I.T. test is computer administered, using high-end graphics and professional voice actors. The C.O.M.I.T. yields standard scores (scores with means equal to 100 and standard deviations of 15) for three scales: Composite IQ, Fluid IQ and Crystallized IQ. This permits direct comparisons with global scores earned on numerous measures of achievement such as the Wechsler series of intelligence tests.
  • The C.O.M.I.T. has a total of 12 sub-tests each producing standard scores (Mean=10; SD=3). Sub-tests that measure fluid intelligence are: Visual Analogies, Visual Closure, Visual Memory, and Auditory Memory. Sub-tests that measure crystallized intelligence are: Vocabulary, Information, and Categorization. The five supplemental scales are: Processing Speed, Social Apperception, Auditory Memory with Visual Distraction, Auditory Memory with Auditory Distraction, and Visual Memory with Auditory Distraction.
  • The selection of which sub-tests to administer during a testing session can be completely controlled. The evaluator has the following broad options:
      • Administer a Screening Battery, consisting of 4 reliable sub-tests and requiring 15-20 minutes to complete.
      • Administer a more comprehensive Standard Battery containing 7 sub-tests and requiring 25-30 minutes to complete.
      • Administer a customized test battery by selecting any or all of the 12 sub-tests.
  • The five supplemental sub-tests not included in either the Standard or the Screening battery requires and additional 3-5 minutes each to administer. Testing may be discontinued between sub-tests and continued at a later time.
  • Composite, Fluid and Crystallized IQ scores are presented in colorful graphic displays. Percentile scores are also presented for these composite scores. Graphs displaying performance on individual sub-tests show scaled scores, as well as percentile scores for each sub-test so that subject performance on each individual task can be measured relative to the normative sample. In addition, a special sub-test scatter score uses sophisticated algorithms to calculate sub-test scatter and summarize the individual's relative strengths and weaknesses compared to their own ipsative mean and overall performance. Percentile scores for sub-test performance are also displayed. All sub-test scores (including supplemental sub-test scores) are presented in colorful bar graphs that allow for clear visual comparisons of performance. In addition, supplemental sub-tests involving distraction yield a Distraction Index that indicates the degree to which an individual's performance was affected by distraction, as well as a visual depiction of which distracting modality proved most troublesome for the subject.
  • While other computer systems may be used, the preferable minimum system requirements are:
      • IBM compatible PC
      • 486 Processor (Pentium recommended)
      • 8 megabytes of RAM (16 Meg or more recommended)
      • 16 bit sound card
      • CD-ROM drive
      • 16-bit Color VGA graphics
  • The present invention also uses a Stress Tolerance Rapid Evaluation Software System (S.T.R.E.S.S.).
  • The ability to remain task focused under pressure is a critical component of successful performance across a wide spectrum of endeavors.
  • The Stress Tolerance Rapid Evaluation Software System (STRESS) is specifically designed to help psychologists and other professionals rapidly evaluate a subject's ability to remain task focused under stressful conditions. It provides a complete picture of the effects of stress on a subject's ability to sustain attention, maintain attention to detail and stay focused on task completion.
  • The STRESS test has wide applicability across diverse areas of practice, ranging from clinical to applied settings. It is unique and extremely precise because it directly measures an individual's ability to stay task focused under stress, rather than attempting to infer it from what an individual is willing to reveal on a self-assessment questionnaire. Some potential uses for the STRESS test include:
      • Candidate screening for high-risk or high-stress jobs
      • Suitability for duty or -re-deployment evaluations when the ability to remain task focused is at issue
      • Stress research where standardized, object measures are required
      • Stress management assessment and treatment evaluation
  • Most individuals can complete the STRESS test in approximately 15 minutes. The test compares a subject's performance on a visual matching task under two conditions: Non-stress and stress. In each condition, subjects are presented with up to 4 items. Each item consists of 3D computer-generated shapes: A target shape and nine potential matches. Subjects must identify all the shapes on the screen that are identical to the target. Items occurring in the same order during non-stress and stress conditions were specifically designed to have identical difficulty-levels. Care was taken to reduce ethnic bias and keep the shapes culture-free. Conditions for producing stress were developed in consultation with experts in the field and specifically designed to induce autonomic nervous system arousal and a sense of time urgency. Professional voice actors do all the voice-overs.
  • For non-stress and stress conditions the following data are reported:
      • The number of items completed during that condition (max=40)
      • The average amount of time required for completing each item
  • Since non-stress and stress items are of identical difficulty, comparisons are made between the items completed during the stress condition and corresponding non-stress items. These comparisons are:
      • The total number of correct matches during each condition
      • The total number of errors of omission or missed opportunities for matches
      • The total number of commission or incorrect matches during each condition
      • The non-stress and stress difference scores for correct matches, errors of omission and errors of commission
  • Examination of the difference-scores from the STRESS test provides a precise indication as to the degree to which an individual is affected by stress, as well as an analysis of the subject's tendency to make errors of commission or errors of omission under pressure.
  • The STRESS test is administered and scored by computer thus eliminating human error introduced by variations in test administration and scoring.
  • While other computer systems may be used, the preferable minimum system requirements are:
      • IBM compatible PC
      • 486 Processor (Pentium recommended)
      • 8 megabytes of RAM (16 Meg or more recommended)
      • 16 bit soundcard
      • CD-ROM drive
      • 16-bit Color VGA graphics
  • It is further noted that other modifications may be made to the present invention, without departing from the scope of the invention, as noted in the appended Claims.

Claims (11)

1. A personnel evaluation system comprising:
at least one multimedia computer for transceiving audio and visual stimuli for a subject;
a storage memory of said system for storing application modules;
at least one central processing unit within said at least one multimedia computer to administer said application modules to said subject, perform analysis of input data, and cause said multimedia computer to transmit said stimuli;
said application modules comprising at least one software application for utilizing multimedia technology to assess in a clinically valid manner respective skills and abilities of the subject which cannot be assessed by non-multimedia test content, said skills and abilities applicable to work-related tasks,; and,
said evaluation system including a software application in which stimuli are presented to a subject, the subject responding to the stimuli as best as he or she can to determine the precise ability level of the subject being evaluated;
wherein said storage memory of said system includes a scoring system derived from database statistics calculated for a representative sample of a specific population of subjects being tested and a predetermined pass/fail threshold for each said application module based on updated database statistics reflecting past test results of prior subjects in a similar job role or in a role requiring similar skills or abilities.
2. The evaluation system of claim 1 in which a subject's performance on a specific task is compared under non-stressful and stressful conditions.
3. The evaluation system of claim 1 including a social perception test in which pictures of people in a variety of postures, facial expressions, and interactions are displayed and an audio presentation of a spoken statement is made, said subject identifying the picture of the person or people most likely to have made that statement or most closely associated with that statement.
4. The evaluation system of claim 1 in which said system moves on to easier sets of questions or more difficult sets of questions when the subject makes a predetermined number of incorrect answers or a predetermined number of correct answers.
5. The evaluation system of claim 1 wherein said system records an answer of a subject to a particular question during a test, updates a test score and dynamically routes subsequent questions in a logical flow to a subject depending upon the correctness or incorrectness of the subject's answer to said particular question, said dynamic routing progressing through said test to provide subsequent questions to the subject depending upon the subject's responses on prior questions of said test to estimate the subject's abilities and dynamically determine the most appropriate starting point and difficulty level of subsequent questions of the test for that subject.
6. The evaluation system of claim 1 in which transmission of said stimuli includes transmission via the internet or a company intranet.
7. A personnel evaluation system comprising:
at least one multimedia computer for transceiving audio and visual stimuli for a subject from at least one of a hard drive of said computer, or the Internet, or a company intranet, or combination thereof;
a storage memory of said system for storing application modules;
at least one central processing unit within said at least one multimedia computer to administer said application modules to said subject, perform analysis of input data, and cause said multimedia computer to transmit said stimuli;
said application modules comprising at least one software application for doing at least one of the following tasks: utilizing multimedia technology to assess skills and abilities in a clinically valid manner, or utilizing multimedia technology to assess skills or abilities applicable to work-related tasks in a more effective manner than how those skills or abilities can be assessed by non-multimedia content or utilizing multimedia technology to compare the performance of a subject on a specific task with distractions present to the performance of the same subject on the same or similar task without distraction present, in a more effective manner than can be assessed when utilizing non-multimedia content or utilizing multimedia technology to compare the performance of a subject on a specific task when under stressful conditions to the same subject's performance on the same or a similar task when under less stressful or non-stressful conditions, in a more effective or real life manner than can be assessed by utilizing non-multimedia content or utilizing multimedia technology to assess the social skills of a subject using audio and facial expressions or social interactions, in a more effective manner than can be assessed by utilizing non-multimedia content, or combinations thereof,
said evaluation system including a software application in which stimuli are presented to a subject, the subject responding to the stimuli as best as he or she can to determine the precise ability level of the subject being evaluated;
wherein said storage memory of said system includes a scoring system derived from database statistics calculated for a representative sample of a specific population being tested and a predetermined pass/fail threshold for each said application module based on updated database statistics reflecting past test results of prior subjects in a similar job role.
8. A personnel evaluation system comprising: a multimedia computer for transceiving audio and visual stimuli for a subject;
a storage memory of said system for storing application modules utilizing multimedia technology to assess skills and abilities in a manner which cannot be assessed by non-multimedia content or utilizing multimedia technology to measure skills or abilities applicable to work-related tasks in a manner which is more clinically valid than the manner in which such skills or abilities must be measured when using only non-multimedia, text-based content;
a central processing unit within said multimedia computer to administer said application modules to said subject, perform analysis of input data, and cause said multimedia computer to transmit said stimuli;
said application modules comprising a software application to complete at least one of the following tasks: to store and evaluate all input data, simulate work-related tasks, deliver stress-inducing audio Or visual stimuli to said subject, said stress-inducing audio or visual stimuli being race, gender, ethnic, and cultural neutral, to interact with said subject, and assess at least one of the intelligence, interest, values, emotional status, mental health, neurological impairment, education, social skills, aptitude, processing speed, performance efficiency, task focus, distractability resistance, stress-tolerance, communication skills, knowledge, reasoning ability, logic skills, job skills, vocabulary, emotional quotient (EQ), impulsivity, risk-tolerance level, personality and specific approach to problem solving;
said evaluation system including a software application in which stimuli are presented to a subject, the subject responding to the stimuli as best as he or she can to determine the precise ability level of the subject being evaluated;
wherein said storage memory of said system includes a. scoring system derived from database statistics calculated for a particular job title, incorporating a suggested default list of application modules to be administered for a specific job title, a suggested weight and signification applied to each said application module score in an overran suitability to job title score, and a predetermined pass/fail threshold for each said application module based on updated database statistics reflecting past test results of prior subjects in a similar job role;
wherein said system incorporates the ability to compare a subject's performance on a task under non-stressful and stressful conditions to determine a subject's stress tolerance level;
wherein said stressful conditions incorporates multimedia stressors using one or more of the following: stressful music, stressful audio, negative feedback, and restricted time deadlines for task completion;
wherein said system incorporates error pattern analysis so as to determine not just the degree to which a subject is affected by stress, but in what manner the subject tends to become affected and in what manner his performance is most likely to change when placed in a stressful environment; and
wherein said system incorporates the ability to automatically stop administration of remaining application modules of said application modules, in the event that the subject's performance is below a predetermined pass/fail threshold for any particular application module of said application modules.
9. A personnel evaluation system comprising:
at least one multimedia computer for transceiving audio and visual stimuli for a subject from at least one of a hard drive of said computer, or the Internet, or a company intranet, or combination thereof;
a storage memory of said system for storing application modules;
at least one central processing unit within said at least one multimedia computer to administer said application modules to said subject, perform analysis of input data, and cause said multimedia computer to transmit said stimuli;
said application modules comprising at least one software application to complete at least one of the following tasks: to store and evaluate all input data, simulate work-related tasks, deliver stress-inducing audio or visual stimuli to said subject, said stress-inducing audio or visual stimuli being race, gender, ethnic, and cultural neutral to interact with said subject, and assess at least one of the intelligence, interest, values, emotional status, mental health, neurological impairment, education, social skills, aptitude, processing speed, performance efficiency, task focus, distractability resistance, stress-tolerance, communication skills, knowledge, reasoning ability, logic skills, job skills, vocabulary, emotional quotient (EQ), impulsivity, risk-tolerance level, personality and specific approach to problem solving;
said evaluation system including a software application in which stimuli are presented to a subject, the subject responding to the stimuli as best as he or she can to determine the precise ability level of the subject being evaluated;
wherein said storage memory of said system includes a scoring system derived from database statistics calculated for a particular job title, incorporating a suggested default list of application modules to be administered for a specific job title, a suggested weight and signification applied to each said application module score in an overall suitability to job title score, and a predetermined pass/fail threshold for each said application module based on updated database statistics reflecting past test results of prior subjects in a particular job role or similar job role; and,
wherein when a scoring ceiling or basal threshold is reached when the subject misses a predetermined number of a consecutive correct answers or answers a predetermined number of consecutive questions correctly in said tests, said system dynamically adapting by automatically moving on to easier or more difficult sets of questions being presented to the subject.
10. A personnel evaluation system comprising:
at least one multimedia computer for transceiving audio and visual stimuli for a subject from at least one of a hard drive of said computer, or the Internet, or a company intranet, or combination thereof;
a storage memory of said system for storing application modules;
at least one central processing unit within said multimedia computer to administer said application modules to said subject, perform analysis of input data, and cause said multimedia computer to transmit said stimuli;
said application modules comprising at least one software application for doing at least one of the following tasks: utilizing multimedia technology to assess skills and abilities in a clinically valid manner or utilizing multimedia technology to assess skills or abilities applicable to work-related tasks in a more effective manner than how those skills or abilities can be assessed by non-multimedia content or utilizing multimedia technology to compare the performance of a subject on a specific task with distractions present to the performance of the same subject on the same or similar task without distraction present, in a more effective manner than can be assessed when utilizing non-multimedia content multimedia technology to compare the performance of a subject on a specific task when under stressful conditions to the same subject's performance on the same or a similar task when under less stressful or non-stressful conditions, in a more effective or real life manner than can be assessed by utilizing non-multimedia content or utilizing multimedia technology to assess the social skills of a subject using audio and facial expressions or social interactions, in a more effective manner than can be assessed by utilizing non-multimedia content, or combinations thereof;
said evaluation system including a software application in which stimuli are presented to a subject, the subject responding to the stimuli as best as he or she can to determine the precise ability level of the subject being evaluated;
wherein said storage memory of said system includes a scoring system derived from database statistics calculated for a particular job title, incorporating a suggested default list of application modules to be administered for a specific job title, a suggested weight and signification applied to each said application module score in an overall suitability to job title score, and a predetermined pass/fail threshold for each said application module based on continually updated database statistics reflecting past test results of prior subjects in a particular job role or similar job role; and
wherein when a scoring ceiling or scoring basal threshold is reached when the subject misses a predetermined number of a consecutive correct answers or answers a predetermined number of consecutive questions correctly in said tests, said system dynamically adapting by automatically moving on to easier or more difficult sets of questions being presented to the subject.
11. A personnel evaluation system comprising:
at least one multimedia computer for transceiving audio and visual stimuli for a subject from at least one of a hard drive of said computer, or the Internet, or a company intranet, or and/or combination thereof;
a storage memory of said system for storing application modules;
at least one central processing unit within said multimedia computer to administer said application modules to said subject, perform analysis of input data, and cause said multimedia computer to transmit said stimuli;
said application modules comprising at least one software application for doing at least one of the following tasks: utilizing multimedia technology to assess skills and abilities in a clinically valid manner or utilizing multimedia technology to assess skills or abilities applicable to work-related tasks in a more effective manner than how those skills or abilities can be assessed by non-multimedia content or utilizing multimedia technology to compare the performance of a subject on a specific task with distractions present to the performance of the same subject on the same or similar task without distraction present, in a more effective manner than can be assessed when utilizing non-multimedia content or utilizing multimedia technology to compare the performance of a subject on a specific task when under stressful conditions to the same subject's performance on the same or a similar task when under less stressful or non-stressful conditions, in a more effective or real life manner than can be assessed by utilizing non-multimedia content or utilizing multimedia technology to assess the social skills of a subject using audio and facial expressions or social interactions, in a more effective manner than can be assessed by utilizing non-multimedia content, or combinations thereof;
said evaluation system including a software application in which stimuli are presented to a subject, the subject responding to the stimuli as best as he or she can to determine the precise ability level of the subject being evaluated;
wherein said storage memory of said system includes a scoring system derived from database statistics calculated for a particular job title, incorporating a suggested default list of application modules to be administered for a specific job title, a suggested weight and signification applied to each said application module score in an overran suitability to job title score, and predetermined pass/fail threshold for each said application module based on updated database statistics reflecting past test results of prior subjects in a similar job role;
wherein said system incorporates the ability to compare a subject's performance on a task under non-stressful and stressful conditions to determine a subject's stress tolerance level;
wherein said stressful conditions incorporates multimedia stressors using one or more of the following: stressful music, stressful audio, negative feedback, and restricted time deadlines for task completion;
wherein said system incorporates error pattern analysis so as to determine not just the degree to which a subject is affected by stress, but in what manner the subject tends to become affected and in what manner his performance is most likely to change when placed in a stressful environment; and
wherein said system incorporates the ability to automatically stop administration of remaining application modules of said application modules, in the event that he subject's performance is below a predetermined pass/fail threshold for any particular application module of said application modules.
US11/645,888 1996-03-27 2006-12-27 Application of multi-media technology to computer administered vocational personnel assessment Abandoned US20070105080A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11/645,888 US20070105080A1 (en) 1996-03-27 2006-12-27 Application of multi-media technology to computer administered vocational personnel assessment

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US1420396P 1996-03-27 1996-03-27
US08/829,488 US6030226A (en) 1996-03-27 1997-03-27 Application of multi-media technology to psychological and educational assessment tools
US09/395,548 US6491525B1 (en) 1996-03-27 1999-09-11 Application of multi-media technology to psychological and educational assessment tools
US27714001P 2001-03-19 2001-03-19
US10/101,227 US7207804B2 (en) 1996-03-27 2002-03-18 Application of multi-media technology to computer administered vocational personnel assessment
US11/645,888 US20070105080A1 (en) 1996-03-27 2006-12-27 Application of multi-media technology to computer administered vocational personnel assessment

Related Parent Applications (2)

Application Number Title Priority Date Filing Date
US09/395,548 Continuation-In-Part US6491525B1 (en) 1996-03-27 1999-09-11 Application of multi-media technology to psychological and educational assessment tools
US10/101,227 Continuation US7207804B2 (en) 1996-03-27 2002-03-18 Application of multi-media technology to computer administered vocational personnel assessment

Publications (1)

Publication Number Publication Date
US20070105080A1 true US20070105080A1 (en) 2007-05-10

Family

ID=27360041

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/101,227 Expired - Fee Related US7207804B2 (en) 1996-03-27 2002-03-18 Application of multi-media technology to computer administered vocational personnel assessment
US11/645,888 Abandoned US20070105080A1 (en) 1996-03-27 2006-12-27 Application of multi-media technology to computer administered vocational personnel assessment

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US10/101,227 Expired - Fee Related US7207804B2 (en) 1996-03-27 2002-03-18 Application of multi-media technology to computer administered vocational personnel assessment

Country Status (1)

Country Link
US (2) US7207804B2 (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050267934A1 (en) * 2004-05-13 2005-12-01 Skillsnet Corporation System and method for defining occupational-specific skills associated with job positions
US20090319327A1 (en) * 2005-12-02 2009-12-24 Netman Co., Ltd. Action improvement system
US20110015497A1 (en) * 2009-07-16 2011-01-20 International Business Machines Corporation System and method to provide career counseling and management using biofeedback
US20110053129A1 (en) * 2009-08-28 2011-03-03 International Business Machines Corporation Adaptive system for real-time behavioral coaching and command intermediation
US20110196802A1 (en) * 2010-02-05 2011-08-11 Nicholas Jeremy Ellis Method and apparatus for hiring using social networks
US20130346332A1 (en) * 2007-05-11 2013-12-26 Agero Connected Services, Inc. Multi-Modal Automation for Human Interactive Skill Assessment
US20140214709A1 (en) * 2013-01-07 2014-07-31 Assessment Innovation, Inc. Occupational performance assessment apparatuses, methods and systems
US20140278832A1 (en) * 2013-03-15 2014-09-18 Abbott Point Of Care Inc. Management system for point of care testing
US20150170303A1 (en) * 2013-12-18 2015-06-18 LifeJourney USA, LLC Methods and systems for providing career inspiration, motivation and guidance to a user
US9971741B2 (en) 2012-12-05 2018-05-15 Chegg, Inc. Authenticated access to accredited testing services
US10055761B2 (en) 2000-07-18 2018-08-21 Google Llc Economic filtering system for delivery of permission based, targeted, incentivized advertising
WO2020106752A1 (en) * 2018-11-21 2020-05-28 Nespa Llc Real-time matching process for job postings based on a system-wide taxonomy
WO2022055473A1 (en) * 2020-09-08 2022-03-17 Shimmy Technologes, Inc. Methods and systems for adaptive apparel design and apparel information architecture

Families Citing this family (107)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6712615B2 (en) * 2000-05-22 2004-03-30 Rolf John Martin High-precision cognitive performance test battery suitable for internet and non-internet use
US20020055870A1 (en) * 2000-06-08 2002-05-09 Thomas Roland R. System for human capital management
US7295991B1 (en) * 2000-11-10 2007-11-13 Erc Dataplus, Inc. Employment sourcing system
JP4462770B2 (en) * 2001-01-31 2010-05-12 富士通株式会社 Adaptive test implementation method
SE0102329D0 (en) * 2001-06-28 2001-06-28 Mapas Syd Ab Control system to achieve quality assured competence development
US20050019734A1 (en) * 2001-11-20 2005-01-27 Avi Peled System and method for diagnosis of mental disorders
US7052277B2 (en) * 2001-12-14 2006-05-30 Kellman A.C.T. Services, Inc. System and method for adaptive learning
US8210850B2 (en) * 2002-03-07 2012-07-03 Blank Marion S Literacy education system for students with autistic spectrum disorders (ASD)
US20030170596A1 (en) * 2002-03-07 2003-09-11 Blank Marion S. Literacy system
US8892895B1 (en) 2002-05-07 2014-11-18 Data Recognition Corporation Integrated system for electronic tracking and control of documents
WO2004023329A1 (en) * 2002-09-09 2004-03-18 Oni Adeboyejo A Enablement of collaborative and integrated decision-making
CA2412109A1 (en) 2002-12-19 2004-06-19 Claude Choquet Virtual simulator method and system for neuromuscular training and certification via a communication network
GB0230291D0 (en) * 2002-12-30 2003-02-05 Human Ecology Ltd System and method for psychological testing
US7258344B2 (en) * 2003-01-16 2007-08-21 Dmi Sports, Inc. Electronically-scored game providing audible feedback and method of use
TW200423696A (en) * 2003-02-04 2004-11-01 Ginganet Corp Remote interview system
US20040191743A1 (en) * 2003-03-26 2004-09-30 International Business Machines Corporation System and method for software development self-assessment
US20040197751A1 (en) * 2003-04-03 2004-10-07 Alexander Albert A. Assessment tool and method for evaluating a person's quality of life
EP1623401A4 (en) * 2003-04-14 2011-07-27 Chorus Inc Human capital management assessment tool system and method
US20050053904A1 (en) * 2003-08-13 2005-03-10 Jennifer Shephard System and method for on-site cognitive efficacy assessment
US20050057801A1 (en) * 2003-09-11 2005-03-17 Bushnell Performance Optics Talking telescope
US20090270717A1 (en) * 2008-04-25 2009-10-29 Welch Allyn, Inc. Apparatus and method for diagnosis of optically identifiable ophthalmic conditions
US7575321B2 (en) 2003-10-30 2009-08-18 Welch Allyn, Inc. Apparatus and method of diagnosis of optically identifiable ophthalmic conditions
US20050114400A1 (en) * 2003-11-24 2005-05-26 Rao Bindu R. Questionnaire network for mobile handsets
US20050154590A1 (en) * 2004-01-09 2005-07-14 Coffey Mark A. Method for assisting the grading and recording of educational tasks in an electronic gradebook using voice recognition
AU2005214713A1 (en) * 2004-02-13 2005-09-01 Emory University Display enhanced testing for concussions and mild traumatic brain injury
US20050244798A1 (en) * 2004-04-30 2005-11-03 Christine Zernzach Method for improving the emotional quotient in infants and children
US20060183099A1 (en) * 2005-02-14 2006-08-17 Feely Richard A Education and test preparation system, method and computer program product
US20060265184A1 (en) * 2005-05-04 2006-11-23 Murray Tondow Method and system for analyzing data for low density sample space
US8517742B1 (en) * 2005-05-17 2013-08-27 American Express Travel Related Services Company, Inc. Labor resource testing system and method
US20070015125A1 (en) * 2005-07-14 2007-01-18 Presidio Sciences, L.P. Automated updating of job assessments
US20070048706A1 (en) * 2005-08-26 2007-03-01 Tan Ah T Psychometric assessment tool and method for interpreting human personality and human behavior
US20070048722A1 (en) * 2005-08-26 2007-03-01 Donald Spector Methods and system for implementing a self-improvement curriculum
US20070065795A1 (en) * 2005-09-21 2007-03-22 Erickson Ranel E Multiple-channel learner-centered whole-brain training system
US9721480B2 (en) * 2006-02-08 2017-08-01 Honeywell International Inc. Augmented tutoring
US20070224580A1 (en) * 2006-02-23 2007-09-27 Mcfaul William J Method for assessing a mental function activity profile
US20070270659A1 (en) * 2006-05-03 2007-11-22 Giegerich Gary D Apparatus and Method for Remotely Detecting Deception
US20080064019A1 (en) * 2006-09-12 2008-03-13 Kaufman Scott M Interactive internet based system for self-assessment and self-improvement
US8135331B2 (en) 2006-11-22 2012-03-13 Bindu Rama Rao System for providing interactive user interactive user interest survey to user of mobile devices
US8700014B2 (en) 2006-11-22 2014-04-15 Bindu Rama Rao Audio guided system for providing guidance to user of mobile device on multi-step activities
US8478250B2 (en) 2007-07-30 2013-07-02 Bindu Rama Rao Interactive media management server
US11256386B2 (en) 2006-11-22 2022-02-22 Qualtrics, Llc Media management system supporting a plurality of mobile devices
US10803474B2 (en) 2006-11-22 2020-10-13 Qualtrics, Llc System for creating and distributing interactive advertisements to mobile devices
US8672843B2 (en) * 2006-11-27 2014-03-18 Qtc Management, Inc. Automated protocol for determining psychiatric disability
US20080160487A1 (en) * 2006-12-29 2008-07-03 Fairfield Language Technologies Modularized computer-aided language learning method and system
JP4242899B2 (en) * 2007-01-19 2009-03-25 株式会社コナミデジタルエンタテインメント Game terminal and program
US20090005654A1 (en) * 2007-03-30 2009-01-01 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Computational user-health testing
US20080287821A1 (en) * 2007-03-30 2008-11-20 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Computational user-health testing
US20090018407A1 (en) * 2007-03-30 2009-01-15 Searete Llc, A Limited Corporation Of The State Of Delaware Computational user-health testing
US20080243543A1 (en) * 2007-03-30 2008-10-02 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Effective response protocols for health monitoring or the like
WO2008122928A1 (en) * 2007-04-04 2008-10-16 Koninklijke Philips Electronics N.V. Determining stress level based on the performance of the person in a game or puzzle
US20080280276A1 (en) * 2007-05-09 2008-11-13 Oregon Health & Science University And Oregon Research Institute Virtual reality tools and techniques for measuring cognitive ability and cognitive impairment
US8699939B2 (en) * 2008-12-19 2014-04-15 Xerox Corporation System and method for recommending educational resources
US8725059B2 (en) * 2007-05-16 2014-05-13 Xerox Corporation System and method for recommending educational resources
US8457544B2 (en) * 2008-12-19 2013-06-04 Xerox Corporation System and method for recommending educational resources
US20100159437A1 (en) * 2008-12-19 2010-06-24 Xerox Corporation System and method for recommending educational resources
WO2009029802A2 (en) * 2007-08-29 2009-03-05 Convergys Gmc Utah Inc. A system and method for screening and rating agents to assess multiple skills as a predictor of agent performance
JP5172305B2 (en) * 2007-12-07 2013-03-27 財団法人 新医療財団 Personality inspection device
US20090164311A1 (en) * 2007-12-19 2009-06-25 Microsoft Corporation Human resource management system
US20100285440A1 (en) * 2007-12-27 2010-11-11 Setu Parikh System and Method to Stimulate Human Genius
US8308484B2 (en) * 2008-02-11 2012-11-13 Rosetta Stone, Ltd. System and methods for detecting deception as to fluency or other ability in a given language
US20090216588A1 (en) * 2008-02-25 2009-08-27 Brian Coyne Return on investment analyzer
US20100075290A1 (en) * 2008-09-25 2010-03-25 Xerox Corporation Automatic Educational Assessment Service
US20100075291A1 (en) * 2008-09-25 2010-03-25 Deyoung Dennis C Automatic educational assessment service
US9737255B2 (en) * 2008-09-30 2017-08-22 National Ict Australia Limited Measuring cognitive load
US20100157345A1 (en) * 2008-12-22 2010-06-24 Xerox Corporation System for authoring educational assessments
US20100178640A1 (en) * 2009-01-12 2010-07-15 Neuro - Technology Solutions Ltd. Computerized test apparatus and methods for quantifying psychological aspects of human responses to stimuli
US20110081640A1 (en) * 2009-10-07 2011-04-07 Hsia-Yen Tseng Systems and Methods for Protecting Websites from Automated Processes Using Visually-Based Children's Cognitive Tests
US20110136085A1 (en) * 2009-12-09 2011-06-09 Gondy Leroy Computer based system and method for assisting an interviewee in remembering and recounting information about a prior event using a cognitive interview and natural language processing
US8768241B2 (en) * 2009-12-17 2014-07-01 Xerox Corporation System and method for representing digital assessments
US20110195389A1 (en) * 2010-02-08 2011-08-11 Xerox Corporation System and method for tracking progression through an educational curriculum
US8521077B2 (en) 2010-07-21 2013-08-27 Xerox Corporation System and method for detecting unauthorized collaboration on educational assessments
US20120271675A1 (en) * 2011-04-19 2012-10-25 Alpine Access, Inc. Dynamic candidate organization system
US9530331B2 (en) * 2011-10-07 2016-12-27 Axeos, LLC System and method for selecting and altering a training session
US20130089851A1 (en) * 2011-10-07 2013-04-11 Axeos, LLC Corporate training system and method for improving workplace performance
US9014614B2 (en) * 2011-10-20 2015-04-21 Cogcubed Corporation Cognitive assessment and treatment platform utilizing a distributed tangible-graphical user interface device
US20130157242A1 (en) * 2011-12-19 2013-06-20 Sanford, L.P. Generating and evaluating learning activities for an educational environment
US9339691B2 (en) 2012-01-05 2016-05-17 Icon Health & Fitness, Inc. System and method for controlling an exercise device
US20130266925A1 (en) * 2012-01-30 2013-10-10 Arizona Board Of Regents On Behalf Of The University Of Arizona Embedded Conversational Agent-Based Kiosk for Automated Interviewing
US9142139B2 (en) * 2012-04-30 2015-09-22 ICON Health& Fitness, Inc. Stimulating learning through exercise
US9207834B2 (en) * 2012-06-11 2015-12-08 Edupresent Llc Layered multimedia interactive assessment system
US20140039975A1 (en) * 2012-08-03 2014-02-06 Sensory Logic, Inc. Emotional modeling of a subject
US9430776B2 (en) 2012-10-25 2016-08-30 Google Inc. Customized E-books
US9009028B2 (en) 2012-12-14 2015-04-14 Google Inc. Custom dictionaries for E-books
EP2969058B1 (en) 2013-03-14 2020-05-13 Icon Health & Fitness, Inc. Strength training apparatus with flywheel and related methods
US9427185B2 (en) * 2013-06-20 2016-08-30 Microsoft Technology Licensing, Llc User behavior monitoring on a computerized device
US9262748B2 (en) 2013-08-15 2016-02-16 International Business Machines Corporation Identifying locations of potential user errors during manipulation of multimedia content
US20150161746A1 (en) * 2013-12-11 2015-06-11 Gerhard H. WENGLORZ Method and apparatus for custom-engineered sourcing
EP3623020A1 (en) 2013-12-26 2020-03-18 Icon Health & Fitness, Inc. Magnetic resistance mechanism in a cable machine
WO2015138339A1 (en) 2014-03-10 2015-09-17 Icon Health & Fitness, Inc. Pressure sensor to quantify work
WO2015191445A1 (en) 2014-06-09 2015-12-17 Icon Health & Fitness, Inc. Cable system incorporated into a treadmill
WO2015195965A1 (en) 2014-06-20 2015-12-23 Icon Health & Fitness, Inc. Post workout massage device
US10391361B2 (en) 2015-02-27 2019-08-27 Icon Health & Fitness, Inc. Simulating real-world terrain on an exercise device
US10116765B2 (en) 2015-07-14 2018-10-30 Tuvi Orbach Needs-matching navigator system
US10839944B2 (en) 2016-02-01 2020-11-17 Tuvi Orbach Computerized interactive psychological assessment protocol—iPAP
US10625137B2 (en) 2016-03-18 2020-04-21 Icon Health & Fitness, Inc. Coordinated displays in an exercise device
US10272317B2 (en) 2016-03-18 2019-04-30 Icon Health & Fitness, Inc. Lighted pace feature in a treadmill
US10493349B2 (en) 2016-03-18 2019-12-03 Icon Health & Fitness, Inc. Display on exercise device
US11189193B2 (en) * 2016-04-26 2021-11-30 Ponddy Education Inc. Affinity knowledge based computational learning system
JP2017227780A (en) * 2016-06-23 2017-12-28 ソニー株式会社 Information processing device, information processing method, and program
US10671705B2 (en) 2016-09-28 2020-06-02 Icon Health & Fitness, Inc. Customizing recipe recommendations
WO2018098406A1 (en) * 2016-11-28 2018-05-31 Intellivance, Llc Multi-sensory literacy acquisition method and system
CN107865665A (en) * 2017-12-15 2018-04-03 安徽徽韵心理咨询有限公司 A kind of psychological test system
CN108742516B (en) * 2018-03-26 2021-03-26 浙江广厦建设职业技术学院 Emotion measuring and adjusting system and method for smart home
US10950010B2 (en) * 2019-01-15 2021-03-16 Square Enix Ltd. Dynamic levels of destructive detail in electronic game display
CN111461153B (en) * 2019-01-22 2023-08-04 刘宏军 Crowd feature deep learning method
CN112396114A (en) * 2020-11-20 2021-02-23 中国科学院深圳先进技术研究院 Evaluation system, evaluation method and related product
US20230305621A1 (en) * 2022-03-22 2023-09-28 Saudi Arabian Oil Company Method and system for managing virtual reality user assessment recordings

Citations (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4671772A (en) * 1985-10-22 1987-06-09 Keilty, Goldsmith & Boone Performance appraisal and training system and method of utilizing same
US4683891A (en) * 1982-04-26 1987-08-04 Vincent Cornellier Biomonitoring stress management method and device
US4931934A (en) * 1988-06-27 1990-06-05 Snyder Thomas E Method and system for measuring clarified intensity of emotion
US5103408A (en) * 1990-01-16 1992-04-07 Atlantic Richfield Company Apparatus and method for determining the ability of an individual to perform a task
US5295491A (en) * 1991-09-26 1994-03-22 Sam Technology, Inc. Non-invasive human neurocognitive performance capability testing method and system
US5326270A (en) * 1991-08-29 1994-07-05 Introspect Technologies, Inc. System and method for assessing an individual's task-processing style
US5344324A (en) * 1992-07-15 1994-09-06 Nova Scientific Corporation Apparatus and method for testing human performance
US5551880A (en) * 1993-01-22 1996-09-03 Bonnstetter; Bill J. Employee success prediction system
US5657256A (en) * 1992-01-31 1997-08-12 Educational Testing Service Method and apparatus for administration of computerized adaptive tests
US5717865A (en) * 1995-09-25 1998-02-10 Stratmann; William C. Method for assisting individuals in decision making processes
US5888074A (en) * 1996-09-16 1999-03-30 Scientex Corporation System for testing and evaluating driver situational awareness
US6227862B1 (en) * 1999-02-12 2001-05-08 Advanced Drivers Education Products And Training, Inc. Driver training system
US6260022B1 (en) * 1994-04-26 2001-07-10 Health Hero Network, Inc. Modular microprocessor-based diagnostic measurement apparatus and method for psychological conditions
US20010042004A1 (en) * 1997-07-02 2001-11-15 Taub Herman P. Methods, systems and apparatuses for matching individuals with behavioral requirements and for managing providers of services to evaluate or increase individuals' behavioral capabilities
US6322368B1 (en) * 1998-07-21 2001-11-27 Cy Research, Inc. Training and testing human judgment of advertising materials
US6425764B1 (en) * 1997-06-09 2002-07-30 Ralph J. Lamson Virtual reality immersion therapy for treating psychological, psychiatric, medical, educational and self-help problems
US6450820B1 (en) * 1999-07-09 2002-09-17 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Method and apparatus for encouraging physiological self-regulation through modulation of an operator's control input to a video game or training simulator
US6503085B1 (en) * 1998-01-29 2003-01-07 James Elkind Use of virtual reality and desk top computer formats to diagnose executive dysfunctions
US20030036683A1 (en) * 2000-05-01 2003-02-20 Kehr Bruce A. Method, system and computer program product for internet-enabled, patient monitoring system
US20030059750A1 (en) * 2000-04-06 2003-03-27 Bindler Paul R. Automated and intelligent networked-based psychological services
US6615182B1 (en) * 1998-05-08 2003-09-02 E-Talk Corporation System and method for defining the organizational structure of an enterprise in a performance evaluation system
US6652283B1 (en) * 1999-12-30 2003-11-25 Cerego, Llc System apparatus and method for maximizing effectiveness and efficiency of learning retaining and retrieving knowledge and skills
US6705869B2 (en) * 2000-06-02 2004-03-16 Darren Schwartz Method and system for interactive communication skill training
US6877989B2 (en) * 2002-02-15 2005-04-12 Psychological Dataccorporation Computer program for generating educational and psychological test items

Family Cites Families (61)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3390397A (en) 1966-02-16 1968-06-25 Bernard Z Friedlander Psychological testing recorder
US3808705A (en) 1971-10-29 1974-05-07 A Schmoyer Instructional apparatus for displaying a changeable still sequence and incrementally advancing a pattern thereon
US4234933A (en) 1978-09-13 1980-11-18 Dimensional Communications, Inc. Electrical game, polling and testing apparatus
US4375080A (en) 1980-06-04 1983-02-22 Barry Patrick D Recording and evaluating instrument and method for teacher evaluation
US4490810A (en) 1982-02-16 1984-12-25 Hon David C Automated instruction, game and data retrieval system
US4684349A (en) 1984-02-15 1987-08-04 Frank Ferguson Audio-visual teaching system and method
DE3529301A1 (en) 1984-08-28 1986-03-27 Jost von Dr. Grüningen Fellenberg Psychotechnical method and device for conducting the method
CH663102A5 (en) 1984-12-21 1987-11-13 Conzelmann Flachglas GLASS PANEL ARRANGEMENT WITH DISC BREAK DETECTOR.
US4760390A (en) 1985-02-25 1988-07-26 Computer Graphics Laboratories, Inc. Graphics display system and method with enhanced instruction data and processing
US4690645A (en) 1985-08-30 1987-09-01 Epoch Company, Ltd. Interactive educational device
US4793810A (en) 1986-11-19 1988-12-27 Data Entry Systems, Inc. Interactive instructional apparatus and method
US4890230A (en) 1986-12-19 1989-12-26 Electric Industry Co., Ltd. Electronic dictionary
US4770636A (en) 1987-04-10 1988-09-13 Albert Einstein College Of Medicine Of Yeshiva University Cognometer
US4839822A (en) 1987-08-13 1989-06-13 501 Synthes (U.S.A.) Computer system and method for suggesting treatments for physical trauma
US4895518A (en) 1987-11-02 1990-01-23 The University Of Michigan Computerized diagnostic reasoning evaluation system
US4810197A (en) 1988-03-24 1989-03-07 Alfonza Hicks Three-dimensional instructional device for teaching physical properties and relationships between basic elements
US5100329A (en) 1990-06-22 1992-03-31 Deesen Kenneth C Computer assisted coaching method
US5102341A (en) 1989-05-05 1992-04-07 Touchstone Applied Science Associates, Inc. Test answer and score sheet device
US5120230A (en) 1989-05-30 1992-06-09 Optical Data Corporation Interactive method for the effective conveyance of information in the form of visual images
US5211564A (en) 1989-07-19 1993-05-18 Educational Testing Service Computerized figural response testing system and method
US5241671C1 (en) 1989-10-26 2002-07-02 Encyclopaedia Britannica Educa Multimedia search system using a plurality of entry path means which indicate interrelatedness of information
US5017142A (en) 1989-11-07 1991-05-21 The United States Of America As Represented By The Secretary Of The Navy Interactive method for testing working memory
US5318450A (en) 1989-11-22 1994-06-07 Gte California Incorporated Multimedia distribution system for instructional materials
US5456607A (en) 1989-12-13 1995-10-10 Antoniak; Peter R. Knowledge testing computer game method employing the repositioning of screen objects to represent data relationships
US5204813A (en) 1990-06-08 1993-04-20 Assessment Systems, Inc. Computer-controlled testing process and device for administering an examination
US5202952A (en) 1990-06-22 1993-04-13 Dragon Systems, Inc. Large-vocabulary continuous speech prefiltering and processing system
JPH04114639A (en) 1990-09-03 1992-04-15 Kyoto Densoku Kk Dementia inspector
US5287489A (en) 1990-10-30 1994-02-15 Hughes Training, Inc. Method and system for authoring, editing and testing instructional materials for use in simulated trailing systems
JPH04241381A (en) 1991-01-16 1992-08-28 Brother Ind Ltd Electronic learning machine
JPH04246791A (en) 1991-02-01 1992-09-02 Hitachi Ltd Questionnaire data collecting and analyzing system
US5437553A (en) 1991-04-08 1995-08-01 Collins; Deborah L. Method and apparatus for automated learning and performance evaluation
EP0513419A1 (en) 1991-05-16 1992-11-19 Robert Dr. Pigache Device for testing the vigilance of a person
CA2044929C (en) 1991-06-18 1997-11-18 Maxime Ferris Teaching method and system
US5240419A (en) 1991-07-15 1993-08-31 Hughes Training, Inc. Condition monitor request processing system
GB9117015D0 (en) 1991-08-07 1991-09-18 Software Solutions Ltd Operation of computer systems
WO1995018565A1 (en) 1991-09-26 1995-07-13 Sam Technology, Inc. Non-invasive neurocognitive testing method and system
US5173051B1 (en) 1991-10-15 1997-06-10 Optical Data Corp Curriculum planning and publishing method
US5379213A (en) 1992-01-28 1995-01-03 Fleetwood Furniture Company, Inc. Test scoring system and method
US5275569A (en) 1992-01-30 1994-01-04 Watkins C Kay Foreign language teaching aid and method
US5267865A (en) 1992-02-11 1993-12-07 John R. Lee Interactive computer aided natural learning method and apparatus
US5302132A (en) 1992-04-01 1994-04-12 Corder Paul R Instructional system and method for improving communication skills
USH1452H (en) 1992-08-25 1995-06-06 Duncan Kennedy Interactive information delivery system and method of training
US5473744A (en) 1992-09-28 1995-12-05 Optical Magnetic Imaging Corporation Computer-assisted interactive method and apparatus for making a multi-media presentation
US5483468A (en) 1992-10-23 1996-01-09 International Business Machines Corporation System and method for concurrent recording and displaying of system performance data
US5377100A (en) 1993-03-08 1994-12-27 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Method of encouraging attention by correlating video game difficulty with attention level
US5696981A (en) 1993-09-30 1997-12-09 Shovers; Aaron H. Personality analyzer
US5454721A (en) 1993-12-30 1995-10-03 Kuch; Nina J. Application of multi-media technology to nutrition education and diet planning
US5540589A (en) 1994-04-11 1996-07-30 Mitsubishi Electric Information Technology Center Audio interactive tutor
AU2365695A (en) 1994-04-26 1995-11-16 Raya Systems, Inc. Modular microprocessor-based diagnostic measurement system for psychological conditions
US5711671A (en) 1994-07-08 1998-01-27 The Board Of Regents Of Oklahoma State University Automated cognitive rehabilitation system and method for treating brain injured patients
US5675817A (en) 1994-12-05 1997-10-07 Motorola, Inc. Language translating pager and method therefor
US5671409A (en) 1995-02-14 1997-09-23 Fatseas; Ted Computer-aided interactive career search system
US5911581A (en) * 1995-02-21 1999-06-15 Braintainment Resources, Inc. Interactive computer program for measuring and analyzing mental ability
US5882203A (en) 1995-05-31 1999-03-16 Correa; Elsa I. Method of detecting depression
US5676551A (en) 1995-09-27 1997-10-14 All Of The Above Inc. Method and apparatus for emotional modulation of a Human personality within the context of an interpersonal relationship
US6029043A (en) * 1998-01-29 2000-02-22 Ho; Chi Fai Computer-aided group-learning methods and systems
US6149586A (en) * 1998-01-29 2000-11-21 Elkind; Jim System and method for diagnosing executive dysfunctions using virtual reality and computer simulation
US6275806B1 (en) * 1999-08-31 2001-08-14 Andersen Consulting, Llp System method and article of manufacture for detecting emotion in voice signals by utilizing statistics for voice signal parameters
US6749432B2 (en) * 1999-10-20 2004-06-15 Impulse Technology Ltd Education system challenging a subject's physiologic and kinesthetic systems to synergistically enhance cognitive function
US6632174B1 (en) 2000-07-06 2003-10-14 Cognifit Ltd (Naiot) Method and apparatus for testing and training cognitive ability
IL157957A0 (en) 2001-03-19 2004-03-28 Techmicro Inc Application of multi-media technology to computer administered vocational personnel assessment

Patent Citations (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4683891A (en) * 1982-04-26 1987-08-04 Vincent Cornellier Biomonitoring stress management method and device
US4671772A (en) * 1985-10-22 1987-06-09 Keilty, Goldsmith & Boone Performance appraisal and training system and method of utilizing same
US4931934A (en) * 1988-06-27 1990-06-05 Snyder Thomas E Method and system for measuring clarified intensity of emotion
US5103408A (en) * 1990-01-16 1992-04-07 Atlantic Richfield Company Apparatus and method for determining the ability of an individual to perform a task
US5326270A (en) * 1991-08-29 1994-07-05 Introspect Technologies, Inc. System and method for assessing an individual's task-processing style
US5295491A (en) * 1991-09-26 1994-03-22 Sam Technology, Inc. Non-invasive human neurocognitive performance capability testing method and system
US5657256A (en) * 1992-01-31 1997-08-12 Educational Testing Service Method and apparatus for administration of computerized adaptive tests
US5344324A (en) * 1992-07-15 1994-09-06 Nova Scientific Corporation Apparatus and method for testing human performance
US5551880A (en) * 1993-01-22 1996-09-03 Bonnstetter; Bill J. Employee success prediction system
US6260022B1 (en) * 1994-04-26 2001-07-10 Health Hero Network, Inc. Modular microprocessor-based diagnostic measurement apparatus and method for psychological conditions
US5717865A (en) * 1995-09-25 1998-02-10 Stratmann; William C. Method for assisting individuals in decision making processes
US5888074A (en) * 1996-09-16 1999-03-30 Scientex Corporation System for testing and evaluating driver situational awareness
US6425764B1 (en) * 1997-06-09 2002-07-30 Ralph J. Lamson Virtual reality immersion therapy for treating psychological, psychiatric, medical, educational and self-help problems
US20010042004A1 (en) * 1997-07-02 2001-11-15 Taub Herman P. Methods, systems and apparatuses for matching individuals with behavioral requirements and for managing providers of services to evaluate or increase individuals' behavioral capabilities
US6341267B1 (en) * 1997-07-02 2002-01-22 Enhancement Of Human Potential, Inc. Methods, systems and apparatuses for matching individuals with behavioral requirements and for managing providers of services to evaluate or increase individuals' behavioral capabilities
US6503085B1 (en) * 1998-01-29 2003-01-07 James Elkind Use of virtual reality and desk top computer formats to diagnose executive dysfunctions
US6615182B1 (en) * 1998-05-08 2003-09-02 E-Talk Corporation System and method for defining the organizational structure of an enterprise in a performance evaluation system
US6322368B1 (en) * 1998-07-21 2001-11-27 Cy Research, Inc. Training and testing human judgment of advertising materials
US6227862B1 (en) * 1999-02-12 2001-05-08 Advanced Drivers Education Products And Training, Inc. Driver training system
US6450820B1 (en) * 1999-07-09 2002-09-17 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Method and apparatus for encouraging physiological self-regulation through modulation of an operator's control input to a video game or training simulator
US6652283B1 (en) * 1999-12-30 2003-11-25 Cerego, Llc System apparatus and method for maximizing effectiveness and efficiency of learning retaining and retrieving knowledge and skills
US20030059750A1 (en) * 2000-04-06 2003-03-27 Bindler Paul R. Automated and intelligent networked-based psychological services
US20030036683A1 (en) * 2000-05-01 2003-02-20 Kehr Bruce A. Method, system and computer program product for internet-enabled, patient monitoring system
US6705869B2 (en) * 2000-06-02 2004-03-16 Darren Schwartz Method and system for interactive communication skill training
US6877989B2 (en) * 2002-02-15 2005-04-12 Psychological Dataccorporation Computer program for generating educational and psychological test items

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
Brauchli et al., Electrocortical, autonomic, and subjective responses to rhythmic audio-visual, 1995, International Journal of Psychophysiology, 19, pp 53-66 *
Segal et al., Clinical Assessment of Object Relations and Social Cognition Using Stories Told to the Picture Arrangement Subtest of the WAIS-R, Journal of Personality Assessment, 1993, 61(1), 58-80 *
U.S. Army Research Institute for the Behavioral and Social Sciences et al., Proceedings, 30th Annual Conference of the Military Testing Association, Arlington, Va., 27 November-2 December 1988, Military Testing Association *
U.S. Army Research Institute for the Behavioral and Social Sciences, Investigating a Background Data Measure of Social Intelligence, March 1995 *
U.S. Army Research Institute for the Behavioral and Social Sciences, Literature Review: Cognitive Abilities - Theory, History, and Validity, February 1991 *

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10269040B2 (en) * 2000-07-18 2019-04-23 Google Llc Economic filtering system for delivery of permission based, targeted, incentivized advertising
US10055761B2 (en) 2000-07-18 2018-08-21 Google Llc Economic filtering system for delivery of permission based, targeted, incentivized advertising
US20050267934A1 (en) * 2004-05-13 2005-12-01 Skillsnet Corporation System and method for defining occupational-specific skills associated with job positions
US20090319327A1 (en) * 2005-12-02 2009-12-24 Netman Co., Ltd. Action improvement system
US8275651B2 (en) * 2005-12-02 2012-09-25 Netman Co., Ltd. System for managing member self-checking of set goal achievement in an organization
US20130346332A1 (en) * 2007-05-11 2013-12-26 Agero Connected Services, Inc. Multi-Modal Automation for Human Interactive Skill Assessment
US9179847B2 (en) * 2009-07-16 2015-11-10 International Business Machines Corporation System and method to provide career counseling and management using biofeedback
US20110015497A1 (en) * 2009-07-16 2011-01-20 International Business Machines Corporation System and method to provide career counseling and management using biofeedback
US9824606B2 (en) * 2009-08-28 2017-11-21 International Business Machines Corporation Adaptive system for real-time behavioral coaching and command intermediation
US20110053129A1 (en) * 2009-08-28 2011-03-03 International Business Machines Corporation Adaptive system for real-time behavioral coaching and command intermediation
US20110196801A1 (en) * 2010-02-05 2011-08-11 Nicholas Jeremy Ellis Method and apparatus for hiring using mobile phones
US20110196802A1 (en) * 2010-02-05 2011-08-11 Nicholas Jeremy Ellis Method and apparatus for hiring using social networks
US10929594B2 (en) 2012-12-05 2021-02-23 Chegg, Inc. Automated testing materials in electronic document publishing
US9971741B2 (en) 2012-12-05 2018-05-15 Chegg, Inc. Authenticated access to accredited testing services
US11741290B2 (en) 2012-12-05 2023-08-29 Chegg, Inc. Automated testing materials in electronic document publishing
US10713415B2 (en) 2012-12-05 2020-07-14 Chegg, Inc. Automated testing materials in electronic document publishing
US10049086B2 (en) 2012-12-05 2018-08-14 Chegg, Inc. Authenticated access to accredited testing services
US10521495B2 (en) 2012-12-05 2019-12-31 Chegg, Inc. Authenticated access to accredited testing services
US11847404B2 (en) 2012-12-05 2023-12-19 Chegg, Inc. Authenticated access to accredited testing services
US11295063B2 (en) 2012-12-05 2022-04-05 Chegg, Inc. Authenticated access to accredited testing services
US10108585B2 (en) * 2012-12-05 2018-10-23 Chegg, Inc. Automated testing materials in electronic document publishing
US20140214709A1 (en) * 2013-01-07 2014-07-31 Assessment Innovation, Inc. Occupational performance assessment apparatuses, methods and systems
US20140278832A1 (en) * 2013-03-15 2014-09-18 Abbott Point Of Care Inc. Management system for point of care testing
US9792572B2 (en) * 2013-03-15 2017-10-17 Abbott Point Of Care Inc. Management system for point of care testing
US10984366B2 (en) 2013-03-15 2021-04-20 Abbott Point Of Care Inc. Management system for point of care testing
US11488088B2 (en) 2013-03-15 2022-11-01 Abbott Point Of Care Inc. Management system for point of care testing
US20150170303A1 (en) * 2013-12-18 2015-06-18 LifeJourney USA, LLC Methods and systems for providing career inspiration, motivation and guidance to a user
US9767525B2 (en) * 2013-12-18 2017-09-19 LifeJourney USA, LLC Methods and systems for providing career inspiration, motivation and guidance to a user
WO2020106752A1 (en) * 2018-11-21 2020-05-28 Nespa Llc Real-time matching process for job postings based on a system-wide taxonomy
WO2022055473A1 (en) * 2020-09-08 2022-03-17 Shimmy Technologes, Inc. Methods and systems for adaptive apparel design and apparel information architecture

Also Published As

Publication number Publication date
US20020106617A1 (en) 2002-08-08
US7207804B2 (en) 2007-04-24

Similar Documents

Publication Publication Date Title
US7207804B2 (en) Application of multi-media technology to computer administered vocational personnel assessment
US20070254270A1 (en) Application of multi-media technology to computer administered personal assessment, self discovery and personal developmental feedback
US6491525B1 (en) Application of multi-media technology to psychological and educational assessment tools
Bagnato Authentic assessment for early childhood intervention: Best practices
US8628331B1 (en) Learning model for competency based performance
Rose et al. Hope measurement
Vrana et al. Can a computer administer a Wechsler Intelligence Test?
Peters et al. Vocabulary size and native speaker self-identification influence flexibility in linguistic prediction among adult bilinguals
Sridhar et al. Going beyond performance scores: Understanding cognitive–affective states in Kindergarteners and application of framework in classrooms
KR100432176B1 (en) Apparatus and method for training using a human interaction simulator
CA2441556C (en) Application of multi-media technology to computer administered vocational personnel assessment
AU2002248670A1 (en) Application of multi-media technology to computer administered vocational personnel assessment
CN113748449A (en) Evaluation and training system
Gallagher et al. 30 Kaufman Assessment Battery for Children
US20060079803A1 (en) Method for improving the presentation of test stimuli during one-on-one clinical psychological tests
Hubal et al. Synthetic characters in health-related applications
Ramezanzade Adding acoustical to visual movement patterns to retest whether imitation is goal-or pattern-directed
Ismayilli Karakoc Teaching medical terminology to speakers of English as a foreign language.
Viljoen Gaze patterns of expert and amateur sight-readers with particular focus on the cognitive underpinnings of reading key and time signatures
Iliopoulos Setting reading accuracy goals affects performance and attitude
Sridhar et al. International Journal of Child-Computer Interaction
Schuerholz The use of technology and media in the assessment of exceptional children
Spichtig Effects of four electronic text presentation formats on reading efficiency and comprehension
Vrana Vrana, SR & Vrana, DT (in press). Can a Computer Administer a Wechsler Intelligence Test? Professional Psychology: Research and Practice.
Pierce Fingerpoint modeling: Effect on picture-governed readers' visual tracking

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION