US20040215452A1 - USB dictation device - Google Patents

USB dictation device Download PDF

Info

Publication number
US20040215452A1
US20040215452A1 US10/424,139 US42413903A US2004215452A1 US 20040215452 A1 US20040215452 A1 US 20040215452A1 US 42413903 A US42413903 A US 42413903A US 2004215452 A1 US2004215452 A1 US 2004215452A1
Authority
US
United States
Prior art keywords
usb
handheld device
button
audio
microphone
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US10/424,139
Other versions
US7263483B2 (en
Inventor
David Pearah
Francis Chen
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nuance Communications Inc
Original Assignee
Dictaphone Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US10/424,139 priority Critical patent/US7263483B2/en
Application filed by Dictaphone Corp filed Critical Dictaphone Corp
Assigned to DICTAPHONE CORPORATION reassignment DICTAPHONE CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHEN, FRANCIS, PEARAH, DAVID E.
Priority to PCT/US2004/012978 priority patent/WO2004098158A2/en
Priority to CA2523599A priority patent/CA2523599C/en
Priority to EP04760411A priority patent/EP1623410A4/en
Publication of US20040215452A1 publication Critical patent/US20040215452A1/en
Assigned to USB AG, STAMFORD BRANCH reassignment USB AG, STAMFORD BRANCH SECURITY AGREEMENT Assignors: NUANCE COMMUNICATIONS, INC.
Assigned to USB AG. STAMFORD BRANCH reassignment USB AG. STAMFORD BRANCH SECURITY AGREEMENT Assignors: NUANCE COMMUNICATIONS, INC.
Priority to US11/830,871 priority patent/US9100742B2/en
Publication of US7263483B2 publication Critical patent/US7263483B2/en
Application granted granted Critical
Assigned to NUANCE COMMUNICATIONS, INC. reassignment NUANCE COMMUNICATIONS, INC. MERGER (SEE DOCUMENT FOR DETAILS). Assignors: DICTAPHONE CORPORATION
Assigned to NUANCE COMMUNICATIONS, INC. reassignment NUANCE COMMUNICATIONS, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: DICTAPHONE CORPORATION
Assigned to MITSUBISH DENKI KABUSHIKI KAISHA, AS GRANTOR, NORTHROP GRUMMAN CORPORATION, A DELAWARE CORPORATION, AS GRANTOR, STRYKER LEIBINGER GMBH & CO., KG, AS GRANTOR, ART ADVANCED RECOGNITION TECHNOLOGIES, INC., A DELAWARE CORPORATION, AS GRANTOR, NUANCE COMMUNICATIONS, INC., AS GRANTOR, SCANSOFT, INC., A DELAWARE CORPORATION, AS GRANTOR, SPEECHWORKS INTERNATIONAL, INC., A DELAWARE CORPORATION, AS GRANTOR, DICTAPHONE CORPORATION, A DELAWARE CORPORATION, AS GRANTOR, HUMAN CAPITAL RESOURCES, INC., A DELAWARE CORPORATION, AS GRANTOR, TELELOGUE, INC., A DELAWARE CORPORATION, AS GRANTOR, DSP, INC., D/B/A DIAMOND EQUIPMENT, A MAINE CORPORATON, AS GRANTOR, NOKIA CORPORATION, AS GRANTOR, INSTITIT KATALIZA IMENI G.K. BORESKOVA SIBIRSKOGO OTDELENIA ROSSIISKOI AKADEMII NAUK, AS GRANTOR reassignment MITSUBISH DENKI KABUSHIKI KAISHA, AS GRANTOR PATENT RELEASE (REEL:018160/FRAME:0909) Assignors: MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT
Assigned to ART ADVANCED RECOGNITION TECHNOLOGIES, INC., A DELAWARE CORPORATION, AS GRANTOR, NUANCE COMMUNICATIONS, INC., AS GRANTOR, SCANSOFT, INC., A DELAWARE CORPORATION, AS GRANTOR, SPEECHWORKS INTERNATIONAL, INC., A DELAWARE CORPORATION, AS GRANTOR, DICTAPHONE CORPORATION, A DELAWARE CORPORATION, AS GRANTOR, TELELOGUE, INC., A DELAWARE CORPORATION, AS GRANTOR, DSP, INC., D/B/A DIAMOND EQUIPMENT, A MAINE CORPORATON, AS GRANTOR reassignment ART ADVANCED RECOGNITION TECHNOLOGIES, INC., A DELAWARE CORPORATION, AS GRANTOR PATENT RELEASE (REEL:017435/FRAME:0199) Assignors: MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT
Adjusted expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00Details of transducers, loudspeakers or microphones
    • H04R1/08Mouthpieces; Microphones; Attachments therefor
    • H04R1/083Special constructions of mouthpieces
    • H04R1/086Protective screens, e.g. all weather or wind screens
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/26Speech to text systems
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2410/00Microphones
    • H04R2410/07Mechanical or electrical reduction of wind noise generated by wind passing a microphone

Definitions

  • Dictation devices have been in common use in many fields in which it is inconvenient or undesirable to make handwritten or typewritten notes.
  • One of the fields in which dictation devices have long been prevalent is the medical profession, and particularly so among radiologists, who often dictate their findings and diagnoses while referring to a photographic print of radiological data such as X-rays. It is common in these fields for a user of a dictation device to make voice recordings and provide these recordings to transcriptionists, who transcribe the recordings in order to generate written transcripts of the recording for the dictator's review or for record keeping purposes.
  • dictation technology has developed significantly and includes such tools as speech recognition software to eliminate some of the need for transcriptionists to transcribe recordings.
  • speech recognition requires high quality audio input. Low quality audio input decreases the effectiveness of speech recognition algorithms, and frequently prevents them from functioning at all. It is therefore desirable to provide a microphone with superior audio quality for connecting to a computer for implementing speech recognition.
  • a dictator uses a dictation device and speech recognition to complete blank text fields in a form.
  • many medical practices have specific forms wherein there is a printed query or prompt, followed by a blank text field into which a practitioner provides the requested text information regarding a particular patient.
  • One way in which different fields are selected and activated to receive text from a voice recognition interface is by use of a pointing device, such as a mouse.
  • a pointing device such as a mouse.
  • this manner of selecting various fields for text input is unduly awkward as typically a user would prefer to use the same hand to manipulate the microphone as the mouse.
  • Dictation microphones for connecting to computers running speech recognition software have been known in the art.
  • these microphones connect to computers through one or more serial ports, and often require other connections as well, including speaker in, audio in, speaker out, audio out, power, RS232, and game port connections.
  • These multiple connections make connecting the microphone to a computer a time consuming and complex process.
  • many dictation microphones require connection to a sound card, which many laptop and some desktop computers lack. Therefore, a substantial number of computers are unable to connect to these sorts of dictation microphones.
  • USB Universal Serial Bus
  • USB2 Universal Serial Bus2
  • a barcode identifier is used to identify the subject about which the user of a dictation device is dictating.
  • a radiologist in the field of radiology, it is common for a radiologist to use a scanner to scan a barcode on a radiograph, such as an X-ray, which identifies the patient and/or the X-ray. The radiologist then records his findings and diagnoses into the dictation device with the assurance that this recording will be associated with the correct patient and/or X-ray. It is therefore desirable to provide a dictation device that may include a barcode or other such scanning ability.
  • an object of the present invention is to provide to provide a microphone with superior audio quality for connecting to a computer for implementing speech recognition.
  • It is another object of the present invention provide an ergonomically convenient way to navigate through forms containing fields in which text is entered through a dictation device and a speech recognition interface.
  • Yet another object of the present invention is to provide a dictation microphone that can be connected simply and easily to a computer through a USB connection, through which two way communication between the microphone and the computer is established, and from which the microphone can draw substantially all of its power requirements.
  • Still another object of the present invention is to provide to provide a dictation device that may include a barcode or other such scanning ability.
  • Another object of the present invention to provide an integrated scanner/dictation device that can interface with a computer through a USB connection which can draw all of its power requirements from the USB port.
  • the invention is a handheld device for receiving audio input in the form of speech, the audio input recorded in a recording medium and processed by a speech recognition engine, thereby generating text.
  • the handheld device may comprise an omni-directional microphone element disposed at a distal end of the device for receiving the audio input in the form of speech and for generating an analog signal therefrom.
  • the handheld device may further comprise a wind screen selected according to a predetermined wind noise sensitivity factor.
  • the wind screen may be acoustically transparent and attenuate wind (air flow) noise for example with hard consonants, that create too great an analog signal and would otherwise disrupt the speech recognition.
  • the handheld device may further comprise an electric circuit for receiving the audio input in the form of speech and for converting the analog signals to digital signals.
  • the electric circuit may have a gain control for providing a signal level that is suitable for speech recognition.
  • the handheld device is preferably used in conjunction with a software program that utilizes a speech recognition engine to fill in blank portions of forms, thus allowing a user to input text in a blank portion of a form by simply speaking the words to be entered in that portion.
  • the handheld device may include a set of input buttons ergonomically positioned on the top surface of the handheld device.
  • the input buttons may include one or more buttons for selectively navigating through predetermined sections of a form, a button for selecting a predetermined section of a form, and a button for initiating recording.
  • the navigation buttons may include a button for advancing to the next predetermined section of a form and a button for going back to a previous predetermined section of a form.
  • the buttons may further include a select button for selecting the active element in a form.
  • the navigation buttons may be used to navigate through various elements in a form by sequentially activating those elements, then when a particular desired element is activated, the select button may be pressed to select the active element. Once an active element has been selected, text may be inserted into that element.
  • the input buttons may include a button for initiating dictation. Preferably, text generated by the speech recognition engine is not immediately displayed as it is recognized because it has been found that for many users, this would be too distracting.
  • the text generated by the speech recognition engine is held in a buffer until the user presses an “insert text” button.
  • the contents of the buffer are displayed in the selected element of the form.
  • the text generated by the speech recognition engine may be displayed immediately after the speech has been recognized and while the user is dictating.
  • buttons may include selecting a portion of text, playing a portion of an audio file, a stop button, a button for reviewing back through an audio file to a previously recorded portion of the file, a button for advancing through an audio file, and a button for playing a portion of an audio file through an integrated speaker on the handheld device.
  • the button for playing a portion of an audio file is also a stop button, such that if any function of the microphone is active, pressing the play/stop button will stop any active function. If no function of the microphone is active, pressing the play/stop button will play a portion of an audio file.
  • the handheld device may also include a button for dictation which when pressed activates the microphone element to receive audio input in the form of speech. When this element is active, the audio input is digitized and transmitted to the speech recognition engine, which then translates the speech into text.
  • the handheld device may also include one or more buttons with programmable functions. These buttons may be assigned a function that may generally be quite complex by way of user recorded or prerecorded macros.
  • the handheld device includes a “signature” button for electronically signing a form after it has been filled out. Actuation of this button designates the current document signed, the current document is saved and further editing is blocked. For example, if the handheld device is being used by a medical doctor, the doctor may complete a form and review it for accuracy, then press the “signature” button, which adds a marker to the document indicating that it has been completed and reviewed, and that the doctor acknowledges that the information contained in the document is true and complete. After the “signature” button is pressed, the document may be saved and marked as read-only to prevent further editing of the document.
  • Another button that would be particularly useful for medical professionals is a “coding” button.
  • a dialog box may appear containing diagnosis numbers.
  • Particular maladies are typically assigned unique codes.
  • a doctor can press the “coding” button and view a menu or other dialog box and select a numerical code corresponding to the diagnosis. This code may then be appended to the document.
  • the numerical codes simplify billing procedures because most insurance carriers rely on these codes rather than a textual description of the maladies treated in order to determine how much to pay for particular procedures.
  • the codes may also provide a database to ease the process of searching, for example, for patients with particular maladies. Further details on coding operations can be found in co-pending U.S. Provisional Patent Application Serial No. 60/436,456, filed Dec. 27, 2002, incorporated herein by reference in its entirety.
  • buttons are ergonomically positioned to allow one-handed navigation through the predetermined sections of a form, selection of a predetermined section of a form, initiation of recording, and insertion of text.
  • the ergonomic positioning of the buttons preferably allows one-handed operation of every function available on the microphone.
  • the handheld device may include a pointing device on its top surface that may be actuated by the user's thumb.
  • the pointing device is a microjoystick or thumbstick, although it may also be a trackball or touchpad, or any other suitable pointing device.
  • the pointing device has associated with it at least one button.
  • the invention is a handheld USB device for receiving audio input in the form of speech.
  • the audio input may be recorded in a recording medium and processed by a speech recognition engine, thereby generating text.
  • the handheld device may include a USB hub for receiving and transmitting signals through a USB interface to a USB root hub in a computer. It may further include a power switch for switching power drawn from the USB interface between a first USB port on the USB hub and a second USB port on the USB hub. It may further include a pointing device connected to the second USB port on the USB hub. It may further include a USB streaming controller connected to the first USB port on the USB hub.
  • the USB streaming controller may receive digital audio signals from an audio codec, which converts analog audio signals from a microphone element on the handheld device into the digital audio signals.
  • the handheld USB device includes a preamplifier between the audio codec and the microphone element for amplifying the analog audio signals from the microphone element before they are digitized by the audio codec into digital audio signals.
  • the handheld USB device also preferably includes a speaker. Digital audio signals from the USB streaming controller may be converted into analog signals by the audio codec. These analog signals may then be amplified by an amplifier, and the amplified analog audio signals then used to drive the speaker.
  • the handheld USB device may include buttons connected to switches, such as USB human interface devices.
  • the buttons and switches may be arranged such that when a button is pressed, its corresponding switch is closed. Alternatively, when a button is pressed, its corresponding switch may be opened, if the default position of the switch is closed.
  • the USB streaming controller may be used to detect whether a switch is opened or closed, and thereby may send a signal through a USB port on the USB hub, which then transmits the signal through a USB interface to a USB root hub on a host computer.
  • the USB streaming controller may also be connected to one or more LEDs, which the USB streaming controller may light in response to a signal indicating that a button has been pressed. Preferably, the USB streaming controller lights an LED when the button corresponding to the “record” function of the handheld USB device is pressed.
  • the handheld USB device may include a data storage device.
  • the data storage device is a memory stick or a SmartMediaTM card.
  • the data storage device may be any data storage means known to those in the art.
  • the handheld USB device may include a fingerprint security device.
  • the fingerprint security device will lock out any unauthorized user, thus preventing unauthorized users to use the handheld USB device to create or alter medical records.
  • the handheld USB device may further include a barcode scanner for scanning barcodes, for example, on medical records to identify the patient, or on pharmaceutical packaging to ensure patients receive the correct medicines.
  • FIG. 1 is a top view of one embodiment of the dictation device of the invention.
  • FIG. 2 is a bottom view of one embodiment of the dictation device of the invention.
  • FIG. 3 is a block diagram of one embodiment of the dictation device of the invention without an integrated scanner.
  • FIG. 4 is a block diagram showing the data flow among the various components of one embodiment of the dictation device of the invention.
  • FIG. 5 is a block diagram of one embodiment of the dictation device of the invention with an integrated scanner.
  • FIG. 6 is a block diagram of one embodiment of the dictation device of the invention with fingerprint security and memory stick features.
  • the present invention encompasses dictation microphones and dictation microphones with integrated scanning units.
  • the invention includes a novel ergonomic layout for facilitating use of the dictation microphones with application software implementing voice recognition algorithms to reduce voice data to text and to place the text in appropriate blank fields in a form.
  • the novel ergonomic layout further includes a pointing device with buttons positioned to facilitate pointing device functionality, such as drag-and-drop.
  • the invention solves a number of problems associated with prior art methods of interfacing dictation devices with computers and provides a convenient USB interface.
  • the invention further solves power limitation problems associated with the integration of dictation devices with scanning devices using a USB interface.
  • the WNS should be less than ⁇ 60 dB at a wind speed of 1 meter/sec in order to provide an audio fidelity adequate for voice recognition. It is anticipated, however, that increasingly sophisticated voice recognition algorithms may overcome the limitations of noise introduced by hard consonant sounds, and that a greater WNS would be adequate.
  • One aspect of this invention is a method for determining whether a particular microphone and windscreen combination is adequate for use with a voice recognition algorithm.
  • FIG. 1 shows a top view of an embodiment of the microphone of the invention 101 .
  • the microphone element 102 may be protected by a windscreen consisting of a layer of foam padding material, preferably polyurethane foam, with a thickness of 0.5 to 1.6 cm and a porosity of between 60 and 90 ppi (pores per inch).
  • the windscreen is a polyurethane foam with a porosity between about 100 and 130 ppi and a thickness of between 0.6 cm and 1 cm, most preferably 0.9 cm.
  • the windscreen minimizes the wind noise sensitivity of the microphone yet has a flat and high transmission profile across the audible frequency range (approximately 20 Hz to 20 kHz).
  • the windscreen is affixed to the microphone by being placed such that its sides are pinched between the microphone casing and the microphone element itself in such a manner as to make it flush with the surface of the microphone.
  • This design prevents shear forces from detaching the windscreen.
  • glue is not used to affix the windscreen to the microphone because the glue could dislodge and get into the microphone element, thus degrading the analog signal from the microphone and making speech recognition difficult or impossible.
  • the microphone 102 is preferably a close-talking microphone.
  • the microphone element is adapted to receive audio input from a speaker whose mouth is between 0.5 and several inches from the microphone element.
  • the microphone element can be as much as 45 degrees of mouth axis.
  • the large variation in frequency response of unidirectional and noise-canceling microphones under those conditions make these microphones ill-suited for this application because of the resulting decrease in the accuracy of speech recognition.
  • these microphones may be used in this application if users are cautious about how they hold the microphone, or if the microphone is mounted in a fixed relation to the user's mouth.
  • the microphone element is preferably an omni-directional microphone with a frequency response that is substantially flat over a range of angles and distances.
  • buttons 104 - 111 allow the user to control the dictation functions.
  • a “next” button 104 allows the user to advance to the next field in a form.
  • a “previous” button 105 allows the user to go back to the previous field in a form.
  • a “select” button 106 allows the user to select the current field in a form.
  • An “insert text” button 107 allows the user to insert text at the current position of the cursor that has been stored in a buffer during dictation.
  • a “dictate” button 108 begins the recording process, allowing the user's speech to be recorded.
  • An LED indicator 103 is lit when the microphone is recording.
  • a “fast forward” button 109 allows the user to skip through previously recorded speech to search for a particular portion.
  • a “review” button 110 allows the user to skip backward through previously recorded speech to search for a particular portion.
  • a combination “stop/play” toggle button either allows the user to play previously recorded speech, or to stop playback of previously recorded speech or any other active function of the microphone.
  • a thumbstick pointing device 112 is included to allow the user to navigate through a form document or for any other reason that a mouse pointing device is normally user for.
  • the thumbstick pointing device also includes a mouse button 113 .
  • the pointing device is a force sensing resistor micro joystick pointing device.
  • the thumbstick pointing device can be any other suitable pointing device, such as a trackball.
  • buttons are optionally present. Generally, these two buttons can be programmable to meet individual users' needs.
  • actuation of one of the additional buttons executes a software routine to provide an electronic signature for signing forms after they have been filled out using the microphone and voice recognition technology.
  • actuation of the signature button adds a “signed” notation to the form after it has been filled out, saves the completed form, and marks it read-only.
  • actuation of the other additional button executes a software routine to provide a menu of codes corresponding to a malady which a doctor has diagnosed in a patient.
  • buttons corresponding to her diagnosis The doctor selects the code corresponding to her diagnosis, and the code may be associated with the form or forwarded to the patient's insurer for billing purposes.
  • Either of these two additional buttons may be programmed to have an arbitrary function.
  • one of the buttons may be programmed to automatically generate and send an email message containing a recently recorded digital audio file recently recorded on the handheld dictation device to a predetermined recipient, as disclosed in co-pending U.S. patent application Ser. No. 09/099,501, entitled “Dictation System Employing Computer-to-Computer Transmission of Voice Files Controlled by Hand Microphone,” filed Jun. 8, 1998, and incorporated herein by reference.
  • FIG. 2 shows a bottom view of an embodiment of the microphone of the invention 201 .
  • a speaker 202 may be included for playing back previously recorded speech or other wav files.
  • a button 203 may be used in conjunction with the thumbstick pointing device 112 and is preferably corresponds to a left mouse button. This allows the user to manipulate the thumbstick pointing device 112 while simultaneously holding down the button 203 , thus allowing for mouse operations such as drag-and-drop.
  • At the base of the microphone 204 may be a slot for removable memory. Also included may be a space 205 for an integrated barcode scanner.
  • FIG. 3 shows a block diagram of an embodiment of the microphone of the invention.
  • a host PC 301 having a USB root hub 302 may be connected to the microphone 303 through a USB connector 305 .
  • the USB connector connects to a USB hub controller 304 in the microphone 303 .
  • the USB hub controller 304 in the microphone 303 can interpret the various signals coming from the elements on board the microphone 303 in order to send those signals in a meaningful way to the host PC 301 .
  • the USB hub controller 304 thus allows multiple elements in the microphone 303 to send signals to the host PC without the signals becoming intractably entangled.
  • the USB hub controller 304 is comprised of a Texas InstrumentsTM TUSB2036 2- or 3-port USB hub chip.
  • a power supply voltage regulator 307 may convert +5V routed from the USB connection to the +3.3V needed to power the USB hub controller chip 304 .
  • the USB hub controller 304 is powered by a Texas InstrumentsTM TPS78833 power supply voltage regulator 307 .
  • a power switch 306 may provide power management for the downstream devices in order to comply with USB power management requirements.
  • the power switch 306 is a Texas InstrumentsTM TPS2044 chip. Both output ports of power switch 306 are tied together, and the +5V output is directed to two power supply voltage regulators 308 and 309 . In the event of an overcurrent, the power switch 306 can switch off the power to the power supply voltage regulators 308 and 309 .
  • the power supply voltage regulators may convert +5V from the USB connection to the +3.3V needed to power the downstream elements.
  • Regulators 308 and 309 may independently provide +3.3V to the digital components of the handheld device (regulator 308 ) and the analog components of the handheld device (regulator 309 ).
  • the independence of the two regulators allows a highly uniform voltage source to be provided to the analog elements (the microphone 314 , speaker 315 , and amplifiers 319 and 320 ), regardless of the power requirements of the digital components (the EEPROM 311 , the audio CODEC 318 , and the streaming controller 312 ).
  • USB hub controller chip is preferably powered by an independent power supply voltage regulator 307 that is not switched by the power switch 306 .
  • the USB streaming controller 312 is attached to a current sensing device or other sensor 321 .
  • the USB streaming controller 312 can send a signal 322 to the speaker amplifier 320 that mutes the speaker 315 , thus reducing the current requirements of the handheld device.
  • the mute signal 322 may still be sent by the host computer 301 when the recording mode is activated in order to prevent sounds from the speaker 315 to be picked up by the microphone 314 during dictation.
  • the dictation microphone 303 has an on-board pointing device, preferably a microjoystick pointing device 310 .
  • the pointing device 310 is connected to one of the ports of the USB hub controller 304 .
  • Microjoysticks that may be used with the present invention can be obtained from Interlink ElectronicsTM.
  • the microphone 303 further includes an electrically erasable programmable read only memory (EEPROM) 311 for storing instructions including CODECs and input and output data bit rates for the USB streaming controller 312 .
  • the EEPROM 311 is connected to the 12 C port of the USB streaming controller 312 .
  • the EEPROM can be programmed or reprogrammed by way of signals from the USB root hub 302 through the USB connection 305 to the USB hub 304 and the USB streaming controller 312 using for example, a device firmware upgrade utility which is compliant with USB Device Class Specification for DFU 1.0.
  • the EEPROM is a Microchip Technology Inc.TM 24LC64 chip.
  • a USB streaming controller 312 receives the input from several switches 313 and the microphone element 314 , and provides the output to the on-board speaker 315 and two light-emitting diodes (LEDs) 316 and 317 .
  • LEDs light-emitting diodes
  • Each of the LEDs may be lit under certain specific circumstances, for example, when the handheld device is plugged in one LED may be lit to indicate the device is ready to be used, and the other LET may be lit during the recording or dictating process.
  • the switches 313 correspond to the buttons 104 - 111 described with reference to FIG. 1, as well as the two programmable buttons, such that actuation of any of those buttons results in the closure of the corresponding switch.
  • the USB streaming controller is a Texas InstrumentsTM TAS 1020A chip.
  • an audio coder/decoder for converting analog signals from the microphone 314 into digital signals that can be transmitted to the PC host 301 or stored in an optional on-board memory (not shown).
  • CODEC audio coder/decoder
  • the audio CODEC 318 also serves the function of converting digital signals from the PC host 301 or stored in an optional on-board memory (not shown) into analog signals, which can be played on the on-board speaker 315 .
  • the audio CODEC 318 is a WolfsonTM WM9707 chip
  • the fixed gain front end amplifier 319 is a National SemiconductorTM LMV 110 chip
  • the speaker amplifier is a 0.35 W mono audio power amplifier comprising a Texas InstrumentsTM TPA301 chip.
  • the microphone 314 is an electric condenser microphone such as Panasonic'sTM WM-52M and the speaker 315 is a PanasonicTM EAS2P104H micro speaker.
  • FIG. 4 is a block diagram showing the data flow through three conceptual “layers”—a function layer 410 , a USB device layer 460 , and a USB bus interface layer 490 .
  • Solid arrows represent data flow and hollow arrows represent logical communications flow, or instructions provided from one component to another.
  • the client software 400 can be any voice recognition software, especially voice recognition software as part of a larger application that allows forms with blank fields to be retrieved and where the blank fields can be filled in using voice recognition.
  • the USB system software 450 and USB host controller 480 can be any suitable USB system and host controller software, for example those provided with typical operating systems having USB capability, such as WindowsTM or Linux.
  • the USB hub 490 is internal to the dictation device, and corresponds to element 304 in FIG. 3 for separate control of a USB Streaming Controller and a pointing device controller.
  • the USB device layer includes a USB streaming controller 470 and the USB pointing device controller 465 , each with a data connection to the USB hub 490 .
  • the function layer includes an audio function 430 , for receiving analog audio data from the microphone physical device and converting the analog data to digital for transmission through the USB streaming controller 470 to the USB hub 490 , through the USB host controller 480 , through the USB system software 450 , to the client software 400 , where voice recognition algorithms converts the audio data to text, and places that text in the appropriate place, for example in a form document.
  • audio function 430 Within the audio function 430 is the capability to output to the speaker physical device.
  • the client software sends digitally encoded audio data through the USB system software 450 , the USB host controller 480 , the USB hub 490 , the USB streaming controller 470 , to the audio function, which converts the digital data to analog, which is then amplified, and the amplified signal drives the speaker physical device.
  • the function layer also includes several input/output (I/O) functions 420 , for receiving input in the form of an identity of an actuated button.
  • This data is likewise transmitted through the USB streaming controller 470 , the USB hub 490 , the USB host controller 480 , the USB system software 450 , to the client software 400 , which recognizes the actuated button as corresponding to a command to be performed in the client software application.
  • the function layer also includes a pointing function 415 , which receives data from the pointing physical device and buttons.
  • the pointing data is routed through the USB micro joystick controller 465 , through the USB hub 490 , through the USB host controller 480 , through the USB system software 450 , to the client software 400 , where the pointing data is used, for example, to navigate through a form having blank fields to be filled in by a user.
  • FIG. 5 is a block diagram of an embodiment of the microphone of the invention having an integrated scanner.
  • the basic design is similar to that shown in FIG. 3, except that there is additionally a scanning device attached to one of the ports 505 of the USB hub.
  • a host PC 301 having a USB root hub 302 may be connected to the microphone 303 through a USB connector 305 .
  • the USB connector connects to a USB hub controller 304 in the microphone 503 .
  • the USB hub controller 304 in the microphone 503 can interpret the various signals coming from the elements on board the microphone 303 in order to send those signals in a meaningful way to the host PC 301 .
  • the USB hub controller 304 thus allows multiple elements in the microphone 303 to send signals to the host PC without the signals becoming intractably entangled.
  • the USB hub controller 304 is comprised of a Texas InstrumentsTM TUSB2036 2- or 3-port USB hub chip.
  • the scan engine 520 may be an integrated device such as the SE923-I00A, which has an RS232 output.
  • the output from the scan engine should be converted from the RS232 standard to the USB standard, for example, by use of an RS232 to USB converter 510 such as an FT8U232AM chip.
  • the USB streaming controller 312 is attached to a current sensing device or other sensor 321 .
  • the sensor 321 senses an overcurrent or the activation of an element with excessive current requirements, for example, when the scanning element 520 is activated, the USB streaming controller 312 can send a signal 322 to the speaker amplifier 320 that mutes the speaker 315 , thus reducing the current requirements of the handheld device.
  • FIG. 6 is a block diagram of one embodiment of the dictation device of the invention with fingerprint security and memory stick features.
  • the basic design of the microphone unit 603 is similar to that shown in FIG. 3, except that there is an additional USB hub controller 610 .
  • the additional USB hub controller may be part of the microphone unit itself, or it may be part of a separate base unit 615 that is attached to the computer 301 .
  • the remainder of the description of this embodiment presumes that there is a separate base unit 615 attached to the computer through the USB root hub 302 .
  • additional supply voltage regulators 650 and an additional power switch 620 provides power to the base unit 615 .
  • the base unit may further have an additional USB hub controller 610 .
  • One part of the additional USB hub controller 610 may be connected to the USB hub controller for the microphone and pointing device 304 , as described above in the text accompanying FIG. 3.
  • the other two ports of the additional USB hub controller 610 may be used for other features.
  • one of the other two ports of the additional USB hub controller 610 may accommodate additional memory, for example, through a memory stick such as SmartMediaTM or any other memory device, so that dictation may be digitally stored within the microphone 603 , base unit 615 , or both.
  • the remaining port of the additional USB hub controller 610 may accommodate a fingerprint identification device 640 .
  • a fingerprint identification device 640 would only allow authorized users to use a microphone of the invention to dictate and sign medical reports.

Abstract

The invention includes a handheld device for receiving audio input in the form of speech. The handheld device is specifically optimized for digitally recording speech input for the purpose of speech recognition. The handheld device includes an ergonomically positioned pointing device to enable dictation and navigation through a document using only one hand. The handheld device may optionally include a memory device, a fingerprint security device, and a barcode scanner.

Description

    BACKGROUND OF THE INVENTION
  • Dictation devices have been in common use in many fields in which it is inconvenient or undesirable to make handwritten or typewritten notes. One of the fields in which dictation devices have long been prevalent is the medical profession, and particularly so among radiologists, who often dictate their findings and diagnoses while referring to a photographic print of radiological data such as X-rays. It is common in these fields for a user of a dictation device to make voice recordings and provide these recordings to transcriptionists, who transcribe the recordings in order to generate written transcripts of the recording for the dictator's review or for record keeping purposes. [0001]
  • More recently, dictation technology has developed significantly and includes such tools as speech recognition software to eliminate some of the need for transcriptionists to transcribe recordings. However, a recognized and pervasive problem in the art is that speech recognition requires high quality audio input. Low quality audio input decreases the effectiveness of speech recognition algorithms, and frequently prevents them from functioning at all. It is therefore desirable to provide a microphone with superior audio quality for connecting to a computer for implementing speech recognition. [0002]
  • In some applications of dictation devices and speech recognition tools, a dictator uses a dictation device and speech recognition to complete blank text fields in a form. For example, many medical practices have specific forms wherein there is a printed query or prompt, followed by a blank text field into which a practitioner provides the requested text information regarding a particular patient. One way in which different fields are selected and activated to receive text from a voice recognition interface is by use of a pointing device, such as a mouse. However, this manner of selecting various fields for text input is unduly awkward as typically a user would prefer to use the same hand to manipulate the microphone as the mouse. Thus, it is desirable to provide an ergonomically convenient way to navigate through forms containing fields in which text is entered through a dictation device and a speech recognition interface. [0003]
  • Technological advances have led to newer and faster types of interfaces between peripheral devices and computers. Dictation microphones for connecting to computers running speech recognition software have been known in the art. However, these microphones connect to computers through one or more serial ports, and often require other connections as well, including speaker in, audio in, speaker out, audio out, power, RS232, and game port connections. These multiple connections make connecting the microphone to a computer a time consuming and complex process. Furthermore, many dictation microphones require connection to a sound card, which many laptop and some desktop computers lack. Therefore, a substantial number of computers are unable to connect to these sorts of dictation microphones. However, the development of the USB and USB2 standards (“Universal Serial Bus” and “USB2” are both hereafter simply referred to as “USB”) have brought significant increases in speed and aided in uniform compatibility between peripheral devices and computers. It is desirable to provide a dictation microphone that can be connected simply and easily to a computer through a USB connection, through which two way communication between the microphone and the computer is established, and from which the microphone can draw substantially all of its power requirements. [0004]
  • In some applications a barcode identifier is used to identify the subject about which the user of a dictation device is dictating. For example, in the field of radiology, it is common for a radiologist to use a scanner to scan a barcode on a radiograph, such as an X-ray, which identifies the patient and/or the X-ray. The radiologist then records his findings and diagnoses into the dictation device with the assurance that this recording will be associated with the correct patient and/or X-ray. It is therefore desirable to provide a dictation device that may include a barcode or other such scanning ability. [0005]
  • A difficulty arises, however, because scanning devices generally, and in particular laser scanning devices, generate electromagnetic fields that can interfere with microphone circuitry and degrade audio signals, thus making speech recognition of those signals less accurate or impossible. Therefore, there is a need for circuitry that can overcome the effects of interference between the scanning and microphone elements in a dictation device having a scanner. It is desirable to have an integrated scanner/dictation device that can interface with a computer through a USB connection, and which can draw all of its power requirements from the USB port. However, the USB standard limits the amount of current that can be drawn to 500 mA, which may be insufficient to drive both the scanner and the dictation device. [0006]
  • OBJECTS OF THE INVENTION
  • In light of the above identified deficiencies of the prior art, an object of the present invention is to provide to provide a microphone with superior audio quality for connecting to a computer for implementing speech recognition. [0007]
  • It is another object of the present invention provide an ergonomically convenient way to navigate through forms containing fields in which text is entered through a dictation device and a speech recognition interface. [0008]
  • Yet another object of the present invention is to provide a dictation microphone that can be connected simply and easily to a computer through a USB connection, through which two way communication between the microphone and the computer is established, and from which the microphone can draw substantially all of its power requirements. [0009]
  • Still another object of the present invention is to provide to provide a dictation device that may include a barcode or other such scanning ability. [0010]
  • Another object of the present invention to provide an integrated scanner/dictation device that can interface with a computer through a USB connection which can draw all of its power requirements from the USB port. [0011]
  • SUMMARY OF THE INVENTION
  • The invention is a handheld device for receiving audio input in the form of speech, the audio input recorded in a recording medium and processed by a speech recognition engine, thereby generating text. The handheld device may comprise an omni-directional microphone element disposed at a distal end of the device for receiving the audio input in the form of speech and for generating an analog signal therefrom. The handheld device may further comprise a wind screen selected according to a predetermined wind noise sensitivity factor. The wind screen may be acoustically transparent and attenuate wind (air flow) noise for example with hard consonants, that create too great an analog signal and would otherwise disrupt the speech recognition. The handheld device may further comprise an electric circuit for receiving the audio input in the form of speech and for converting the analog signals to digital signals. The electric circuit may have a gain control for providing a signal level that is suitable for speech recognition. [0012]
  • The handheld device is preferably used in conjunction with a software program that utilizes a speech recognition engine to fill in blank portions of forms, thus allowing a user to input text in a blank portion of a form by simply speaking the words to be entered in that portion. The handheld device may include a set of input buttons ergonomically positioned on the top surface of the handheld device. The input buttons may include one or more buttons for selectively navigating through predetermined sections of a form, a button for selecting a predetermined section of a form, and a button for initiating recording. [0013]
  • The navigation buttons may include a button for advancing to the next predetermined section of a form and a button for going back to a previous predetermined section of a form. The buttons may further include a select button for selecting the active element in a form. The navigation buttons may be used to navigate through various elements in a form by sequentially activating those elements, then when a particular desired element is activated, the select button may be pressed to select the active element. Once an active element has been selected, text may be inserted into that element. The input buttons may include a button for initiating dictation. Preferably, text generated by the speech recognition engine is not immediately displayed as it is recognized because it has been found that for many users, this would be too distracting. Thus when dictation has been initiated by the user by pressing the dictation button, the text generated by the speech recognition engine is held in a buffer until the user presses an “insert text” button. When the user presses the “insert text” button, the contents of the buffer are displayed in the selected element of the form. In an alternative embodiment, the text generated by the speech recognition engine may be displayed immediately after the speech has been recognized and while the user is dictating. [0014]
  • Other functions of the buttons may include selecting a portion of text, playing a portion of an audio file, a stop button, a button for reviewing back through an audio file to a previously recorded portion of the file, a button for advancing through an audio file, and a button for playing a portion of an audio file through an integrated speaker on the handheld device. In one embodiment, the button for playing a portion of an audio file is also a stop button, such that if any function of the microphone is active, pressing the play/stop button will stop any active function. If no function of the microphone is active, pressing the play/stop button will play a portion of an audio file. The handheld device may also include a button for dictation which when pressed activates the microphone element to receive audio input in the form of speech. When this element is active, the audio input is digitized and transmitted to the speech recognition engine, which then translates the speech into text. [0015]
  • The handheld device may also include one or more buttons with programmable functions. These buttons may be assigned a function that may generally be quite complex by way of user recorded or prerecorded macros. In a preferred embodiment, the handheld device includes a “signature” button for electronically signing a form after it has been filled out. Actuation of this button designates the current document signed, the current document is saved and further editing is blocked. For example, if the handheld device is being used by a medical doctor, the doctor may complete a form and review it for accuracy, then press the “signature” button, which adds a marker to the document indicating that it has been completed and reviewed, and that the doctor acknowledges that the information contained in the document is true and complete. After the “signature” button is pressed, the document may be saved and marked as read-only to prevent further editing of the document. [0016]
  • Another button that would be particularly useful for medical professionals is a “coding” button. When the “coding” button is pressed, a dialog box may appear containing diagnosis numbers. Particular maladies are typically assigned unique codes. A doctor can press the “coding” button and view a menu or other dialog box and select a numerical code corresponding to the diagnosis. This code may then be appended to the document. The numerical codes simplify billing procedures because most insurance carriers rely on these codes rather than a textual description of the maladies treated in order to determine how much to pay for particular procedures. The codes may also provide a database to ease the process of searching, for example, for patients with particular maladies. Further details on coding operations can be found in co-pending U.S. Provisional Patent Application Serial No. 60/436,456, filed Dec. 27, 2002, incorporated herein by reference in its entirety. [0017]
  • Preferably, the buttons are ergonomically positioned to allow one-handed navigation through the predetermined sections of a form, selection of a predetermined section of a form, initiation of recording, and insertion of text. The ergonomic positioning of the buttons preferably allows one-handed operation of every function available on the microphone. [0018]
  • The handheld device may include a pointing device on its top surface that may be actuated by the user's thumb. Preferably, the pointing device is a microjoystick or thumbstick, although it may also be a trackball or touchpad, or any other suitable pointing device. The pointing device has associated with it at least one button. Preferably, there is a button on the top of the handheld device, just below the pointing device. Most preferably, there is a button on the bottom of the handheld device, beneath where a user's index finger would naturally rest, such that advanced mouse maneuvers such as drag-and-drop may be performed by holding down the button with the index finger while moving the pointing device with the thumb. [0019]
  • In one aspect, the invention is a handheld USB device for receiving audio input in the form of speech. The audio input may be recorded in a recording medium and processed by a speech recognition engine, thereby generating text. The handheld device may include a USB hub for receiving and transmitting signals through a USB interface to a USB root hub in a computer. It may further include a power switch for switching power drawn from the USB interface between a first USB port on the USB hub and a second USB port on the USB hub. It may further include a pointing device connected to the second USB port on the USB hub. It may further include a USB streaming controller connected to the first USB port on the USB hub. The USB streaming controller may receive digital audio signals from an audio codec, which converts analog audio signals from a microphone element on the handheld device into the digital audio signals. [0020]
  • Preferably, the handheld USB device includes a preamplifier between the audio codec and the microphone element for amplifying the analog audio signals from the microphone element before they are digitized by the audio codec into digital audio signals. The handheld USB device also preferably includes a speaker. Digital audio signals from the USB streaming controller may be converted into analog signals by the audio codec. These analog signals may then be amplified by an amplifier, and the amplified analog audio signals then used to drive the speaker. [0021]
  • The handheld USB device may include buttons connected to switches, such as USB human interface devices. The buttons and switches may be arranged such that when a button is pressed, its corresponding switch is closed. Alternatively, when a button is pressed, its corresponding switch may be opened, if the default position of the switch is closed. The USB streaming controller may be used to detect whether a switch is opened or closed, and thereby may send a signal through a USB port on the USB hub, which then transmits the signal through a USB interface to a USB root hub on a host computer. The USB streaming controller may also be connected to one or more LEDs, which the USB streaming controller may light in response to a signal indicating that a button has been pressed. Preferably, the USB streaming controller lights an LED when the button corresponding to the “record” function of the handheld USB device is pressed. [0022]
  • In one embodiment, the handheld USB device may include a data storage device. Preferably, the data storage device is a memory stick or a SmartMedia™ card. In other embodiments, the data storage device may be any data storage means known to those in the art. In another embodiment, the handheld USB device may include a fingerprint security device. Preferably, the fingerprint security device will lock out any unauthorized user, thus preventing unauthorized users to use the handheld USB device to create or alter medical records. The handheld USB device may further include a barcode scanner for scanning barcodes, for example, on medical records to identify the patient, or on pharmaceutical packaging to ensure patients receive the correct medicines. [0023]
  • The above advantages and features are of representative embodiments only, and are presented only to assist in understanding the invention. It should be understood that they are not to be considered limitations on the invention as defined by the claims, or limitations on equivalents to the claims. Additional features and advantages of the invention will become apparent from the drawings, the following description, and the claims. [0024]
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • While the specification concludes with claims particularly pointing out and distinctly claiming the present invention, it is believed the same will be better understood from the following description taken in conjunction with the accompanying drawings, which illustrate, in a non-limiting fashion, the best mode presently contemplated for carrying out the present invention, and in which like reference numerals designate like parts throughout the figures, wherein: [0025]
  • FIG. 1 is a top view of one embodiment of the dictation device of the invention. [0026]
  • FIG. 2 is a bottom view of one embodiment of the dictation device of the invention. [0027]
  • FIG. 3 is a block diagram of one embodiment of the dictation device of the invention without an integrated scanner. [0028]
  • FIG. 4 is a block diagram showing the data flow among the various components of one embodiment of the dictation device of the invention. [0029]
  • FIG. 5 is a block diagram of one embodiment of the dictation device of the invention with an integrated scanner. [0030]
  • FIG. 6 is a block diagram of one embodiment of the dictation device of the invention with fingerprint security and memory stick features. [0031]
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • The present invention encompasses dictation microphones and dictation microphones with integrated scanning units. The invention includes a novel ergonomic layout for facilitating use of the dictation microphones with application software implementing voice recognition algorithms to reduce voice data to text and to place the text in appropriate blank fields in a form. The novel ergonomic layout further includes a pointing device with buttons positioned to facilitate pointing device functionality, such as drag-and-drop. The invention solves a number of problems associated with prior art methods of interfacing dictation devices with computers and provides a convenient USB interface. The invention further solves power limitation problems associated with the integration of dictation devices with scanning devices using a USB interface. [0032]
  • It is known in the art that hard consonant sounds, like -d, -t, -p, -b, -x, and -z have higher instantaneous energies, that is they create higher wind speed than other sounds, and can reduce a microphone's fidelity, and thus cause speech recognition rates to drop. This effect is deemed “Wind Noise Sensitivity” (WNS). The inventors have developed a test for determining whether a microphone and windscreen combination is adequately resistant to these wind noises. For an electret microphone with an output sensitivity of −44 dB (1 KHz, 0 dB=1 V/pA), the WNS should be less than −60 dB at a wind speed of 1 meter/sec in order to provide an audio fidelity adequate for voice recognition. It is anticipated, however, that increasingly sophisticated voice recognition algorithms may overcome the limitations of noise introduced by hard consonant sounds, and that a greater WNS would be adequate. One aspect of this invention is a method for determining whether a particular microphone and windscreen combination is adequate for use with a voice recognition algorithm. [0033]
  • FIG. 1 shows a top view of an embodiment of the microphone of the [0034] invention 101. The microphone element 102 may be protected by a windscreen consisting of a layer of foam padding material, preferably polyurethane foam, with a thickness of 0.5 to 1.6 cm and a porosity of between 60 and 90 ppi (pores per inch). Preferably, the windscreen is a polyurethane foam with a porosity between about 100 and 130 ppi and a thickness of between 0.6 cm and 1 cm, most preferably 0.9 cm. Most preferably, the windscreen minimizes the wind noise sensitivity of the microphone yet has a flat and high transmission profile across the audible frequency range (approximately 20 Hz to 20 kHz). Preferably, the windscreen is affixed to the microphone by being placed such that its sides are pinched between the microphone casing and the microphone element itself in such a manner as to make it flush with the surface of the microphone. This design prevents shear forces from detaching the windscreen. Preferably, glue is not used to affix the windscreen to the microphone because the glue could dislodge and get into the microphone element, thus degrading the analog signal from the microphone and making speech recognition difficult or impossible.
  • The [0035] microphone 102 is preferably a close-talking microphone. Preferably, the microphone element is adapted to receive audio input from a speaker whose mouth is between 0.5 and several inches from the microphone element. Preferably, the microphone element can be as much as 45 degrees of mouth axis. The large variation in frequency response of unidirectional and noise-canceling microphones under those conditions make these microphones ill-suited for this application because of the resulting decrease in the accuracy of speech recognition. However, these microphones may be used in this application if users are cautious about how they hold the microphone, or if the microphone is mounted in a fixed relation to the user's mouth. The microphone element is preferably an omni-directional microphone with a frequency response that is substantially flat over a range of angles and distances.
  • A plurality of buttons [0036] 104-111 allow the user to control the dictation functions. A “next” button 104 allows the user to advance to the next field in a form. A “previous” button 105 allows the user to go back to the previous field in a form. A “select” button 106 allows the user to select the current field in a form. An “insert text” button 107 allows the user to insert text at the current position of the cursor that has been stored in a buffer during dictation. A “dictate” button 108 begins the recording process, allowing the user's speech to be recorded. An LED indicator 103 is lit when the microphone is recording. A “fast forward” button 109 allows the user to skip through previously recorded speech to search for a particular portion. A “review” button 110 allows the user to skip backward through previously recorded speech to search for a particular portion. A combination “stop/play” toggle button either allows the user to play previously recorded speech, or to stop playback of previously recorded speech or any other active function of the microphone.
  • A [0037] thumbstick pointing device 112 is included to allow the user to navigate through a form document or for any other reason that a mouse pointing device is normally user for. The thumbstick pointing device also includes a mouse button 113. In a preferred embodiment, the pointing device is a force sensing resistor micro joystick pointing device. In alternative embodiments, the thumbstick pointing device can be any other suitable pointing device, such as a trackball.
  • Two additional buttons (not shown) are optionally present. Generally, these two buttons can be programmable to meet individual users' needs. In a preferred embodiment, actuation of one of the additional buttons executes a software routine to provide an electronic signature for signing forms after they have been filled out using the microphone and voice recognition technology. In this embodiment, actuation of the signature button adds a “signed” notation to the form after it has been filled out, saves the completed form, and marks it read-only. In this embodiment, actuation of the other additional button executes a software routine to provide a menu of codes corresponding to a malady which a doctor has diagnosed in a patient. The doctor selects the code corresponding to her diagnosis, and the code may be associated with the form or forwarded to the patient's insurer for billing purposes. Either of these two additional buttons may be programmed to have an arbitrary function. For example, one of the buttons may be programmed to automatically generate and send an email message containing a recently recorded digital audio file recently recorded on the handheld dictation device to a predetermined recipient, as disclosed in co-pending U.S. patent application Ser. No. 09/099,501, entitled “Dictation System Employing Computer-to-Computer Transmission of Voice Files Controlled by Hand Microphone,” filed Jun. 8, 1998, and incorporated herein by reference. [0038]
  • FIG. 2 shows a bottom view of an embodiment of the microphone of the [0039] invention 201. A speaker 202 may be included for playing back previously recorded speech or other wav files. A button 203 may be used in conjunction with the thumbstick pointing device 112 and is preferably corresponds to a left mouse button. This allows the user to manipulate the thumbstick pointing device 112 while simultaneously holding down the button 203, thus allowing for mouse operations such as drag-and-drop. At the base of the microphone 204 may be a slot for removable memory. Also included may be a space 205 for an integrated barcode scanner.
  • FIG. 3 shows a block diagram of an embodiment of the microphone of the invention. In this embodiment, a [0040] host PC 301 having a USB root hub 302 may be connected to the microphone 303 through a USB connector 305. The USB connector connects to a USB hub controller 304 in the microphone 303. The USB hub controller 304 in the microphone 303 can interpret the various signals coming from the elements on board the microphone 303 in order to send those signals in a meaningful way to the host PC 301. The USB hub controller 304 thus allows multiple elements in the microphone 303 to send signals to the host PC without the signals becoming intractably entangled. In a preferred embodiment, the USB hub controller 304 is comprised of a Texas Instruments™ TUSB2036 2- or 3-port USB hub chip. A power supply voltage regulator 307 may convert +5V routed from the USB connection to the +3.3V needed to power the USB hub controller chip 304. In a preferred embodiment, the USB hub controller 304 is powered by a Texas Instruments™ TPS78833 power supply voltage regulator 307.
  • A [0041] power switch 306 may provide power management for the downstream devices in order to comply with USB power management requirements. Preferably, the power switch 306 is a Texas Instruments™ TPS2044 chip. Both output ports of power switch 306 are tied together, and the +5V output is directed to two power supply voltage regulators 308 and 309. In the event of an overcurrent, the power switch 306 can switch off the power to the power supply voltage regulators 308 and 309. When the power switch 306 is active, thus sending +5V to the power supply voltage regulators 308 and 309, the power supply voltage regulators may convert +5V from the USB connection to the +3.3V needed to power the downstream elements. Regulators 308 and 309 may independently provide +3.3V to the digital components of the handheld device (regulator 308) and the analog components of the handheld device (regulator 309). The independence of the two regulators allows a highly uniform voltage source to be provided to the analog elements (the microphone 314, speaker 315, and amplifiers 319 and 320), regardless of the power requirements of the digital components (the EEPROM 311, the audio CODEC 318, and the streaming controller 312).
  • Under the USB standard, only 500 mA of current can be drawn from the USB connection. Preferably, high-power functions will draw less than 100 mA at power up. Thus, the [0042] power switch 306 serves the additional function of shutting down the downstream devices if they attempt to draw more than this maximum amount of current. Note that the power to the USB hub controller chip is preferably powered by an independent power supply voltage regulator 307 that is not switched by the power switch 306. In a preferred embodiment, the USB streaming controller 312 is attached to a current sensing device or other sensor 321. When the sensor 321 senses an overcurrent or the activation of an element with excessive current requirements, the USB streaming controller 312 can send a signal 322 to the speaker amplifier 320 that mutes the speaker 315, thus reducing the current requirements of the handheld device. In the embodiment of the invention shown in FIG. 3, there is no barcode scanner, and thus it is not expected that a current in excess of 500 mA would ever be required. Thus the sensor element 321 is not strictly necessary in this embodiment. However, the mute signal 322 may still be sent by the host computer 301 when the recording mode is activated in order to prevent sounds from the speaker 315 to be picked up by the microphone 314 during dictation.
  • In a preferred embodiment, the [0043] dictation microphone 303 has an on-board pointing device, preferably a microjoystick pointing device 310. The pointing device 310 is connected to one of the ports of the USB hub controller 304. Microjoysticks that may be used with the present invention can be obtained from Interlink Electronics™.
  • The [0044] microphone 303 further includes an electrically erasable programmable read only memory (EEPROM) 311 for storing instructions including CODECs and input and output data bit rates for the USB streaming controller 312. The EEPROM 311 is connected to the 12C port of the USB streaming controller 312. The EEPROM can be programmed or reprogrammed by way of signals from the USB root hub 302 through the USB connection 305 to the USB hub 304 and the USB streaming controller 312 using for example, a device firmware upgrade utility which is compliant with USB Device Class Specification for DFU 1.0. Preferably, the EEPROM is a Microchip Technology Inc.™ 24LC64 chip.
  • A [0045] USB streaming controller 312 receives the input from several switches 313 and the microphone element 314, and provides the output to the on-board speaker 315 and two light-emitting diodes (LEDs) 316 and 317. Each of the LEDs may be lit under certain specific circumstances, for example, when the handheld device is plugged in one LED may be lit to indicate the device is ready to be used, and the other LET may be lit during the recording or dictating process. The switches 313 correspond to the buttons 104-111 described with reference to FIG. 1, as well as the two programmable buttons, such that actuation of any of those buttons results in the closure of the corresponding switch. In a preferred embodiment, the USB streaming controller is a Texas Instruments™ TAS 1020A chip.
  • Also connected to the [0046] USB streaming controller 312 is an audio coder/decoder (CODEC) for converting analog signals from the microphone 314 into digital signals that can be transmitted to the PC host 301 or stored in an optional on-board memory (not shown). Between the microphone 314 and the audio CODEC 318 is a fixed gain front end amplifier 319 for amplifying the analog signal from the microphone to take advantage of a greater dynamic range before the signal is digitized. The audio CODEC 318 also serves the function of converting digital signals from the PC host 301 or stored in an optional on-board memory (not shown) into analog signals, which can be played on the on-board speaker 315. Between the audio CODEC 318 and the speaker 315 is an amplifier 320 for amplifying the analog signal to produce an adequate volume at the speaker 315. In a preferred embodiment, the audio CODEC 318 is a Wolfson™ WM9707 chip, the fixed gain front end amplifier 319 is a National Semiconductor™ LMV 110 chip, and the speaker amplifier is a 0.35 W mono audio power amplifier comprising a Texas Instruments™ TPA301 chip. Also in this preferred embodiment, the microphone 314 is an electric condenser microphone such as Panasonic's™ WM-52M and the speaker 315 is a Panasonic™ EAS2P104H micro speaker.
  • FIG. 4 is a block diagram showing the data flow through three conceptual “layers”—a [0047] function layer 410, a USB device layer 460, and a USB bus interface layer 490. Solid arrows represent data flow and hollow arrows represent logical communications flow, or instructions provided from one component to another. The client software 400 can be any voice recognition software, especially voice recognition software as part of a larger application that allows forms with blank fields to be retrieved and where the blank fields can be filled in using voice recognition. The USB system software 450 and USB host controller 480 can be any suitable USB system and host controller software, for example those provided with typical operating systems having USB capability, such as Windows™ or Linux. The USB hub 490 is internal to the dictation device, and corresponds to element 304 in FIG. 3 for separate control of a USB Streaming Controller and a pointing device controller. The USB device layer includes a USB streaming controller 470 and the USB pointing device controller 465, each with a data connection to the USB hub 490.
  • The function layer includes an [0048] audio function 430, for receiving analog audio data from the microphone physical device and converting the analog data to digital for transmission through the USB streaming controller 470 to the USB hub 490, through the USB host controller 480, through the USB system software 450, to the client software 400, where voice recognition algorithms converts the audio data to text, and places that text in the appropriate place, for example in a form document. Within the audio function 430 is the capability to output to the speaker physical device. The client software sends digitally encoded audio data through the USB system software 450, the USB host controller 480, the USB hub 490, the USB streaming controller 470, to the audio function, which converts the digital data to analog, which is then amplified, and the amplified signal drives the speaker physical device.
  • The function layer also includes several input/output (I/O) functions [0049] 420, for receiving input in the form of an identity of an actuated button. This data is likewise transmitted through the USB streaming controller 470, the USB hub 490, the USB host controller 480, the USB system software 450, to the client software 400, which recognizes the actuated button as corresponding to a command to be performed in the client software application.
  • The function layer also includes a [0050] pointing function 415, which receives data from the pointing physical device and buttons. The pointing data is routed through the USB micro joystick controller 465, through the USB hub 490, through the USB host controller 480, through the USB system software 450, to the client software 400, where the pointing data is used, for example, to navigate through a form having blank fields to be filled in by a user.
  • FIG. 5 is a block diagram of an embodiment of the microphone of the invention having an integrated scanner. The basic design is similar to that shown in FIG. 3, except that there is additionally a scanning device attached to one of the [0051] ports 505 of the USB hub. A host PC 301 having a USB root hub 302 may be connected to the microphone 303 through a USB connector 305. The USB connector connects to a USB hub controller 304 in the microphone 503. The USB hub controller 304 in the microphone 503 can interpret the various signals coming from the elements on board the microphone 303 in order to send those signals in a meaningful way to the host PC 301. The USB hub controller 304 thus allows multiple elements in the microphone 303 to send signals to the host PC without the signals becoming intractably entangled. In a preferred embodiment, the USB hub controller 304 is comprised of a Texas Instruments™ TUSB2036 2- or 3-port USB hub chip.
  • The [0052] scan engine 520 may be an integrated device such as the SE923-I00A, which has an RS232 output. The output from the scan engine should be converted from the RS232 standard to the USB standard, for example, by use of an RS232 to USB converter 510 such as an FT8U232AM chip. In a preferred embodiment, the USB streaming controller 312 is attached to a current sensing device or other sensor 321. When the sensor 321 senses an overcurrent or the activation of an element with excessive current requirements, for example, when the scanning element 520 is activated, the USB streaming controller 312 can send a signal 322 to the speaker amplifier 320 that mutes the speaker 315, thus reducing the current requirements of the handheld device.
  • FIG. 6 is a block diagram of one embodiment of the dictation device of the invention with fingerprint security and memory stick features. The basic design of the [0053] microphone unit 603 is similar to that shown in FIG. 3, except that there is an additional USB hub controller 610. The additional USB hub controller may be part of the microphone unit itself, or it may be part of a separate base unit 615 that is attached to the computer 301. The remainder of the description of this embodiment presumes that there is a separate base unit 615 attached to the computer through the USB root hub 302. In this embodiment, additional supply voltage regulators 650 and an additional power switch 620 provides power to the base unit 615.
  • The base unit may further have an additional USB hub controller [0054] 610. One part of the additional USB hub controller 610 may be connected to the USB hub controller for the microphone and pointing device 304, as described above in the text accompanying FIG. 3. The other two ports of the additional USB hub controller 610 may be used for other features. In this embodiment, one of the other two ports of the additional USB hub controller 610 may accommodate additional memory, for example, through a memory stick such as SmartMedia™ or any other memory device, so that dictation may be digitally stored within the microphone 603, base unit 615, or both. The remaining port of the additional USB hub controller 610 may accommodate a fingerprint identification device 640. A fingerprint identification device 640 would only allow authorized users to use a microphone of the invention to dictate and sign medical reports.

Claims (27)

What is claimed is:
1. A handheld device for receiving audio input in the form of speech, the audio input recorded in a recording medium and processed by a speech recognition engine, thereby generating text, the handheld device comprising:
an omni-directional microphone element disposed at a distal end,
a wind screen selected according to a predetermined wind noise sensitivity factor,
an electric circuit for receiving the audio input in the form of speech and for converting analog signals to digital signals, the electric circuit having a gain control for providing a signal level that is suitable for speech recognition, and
a set of input buttons ergonomically positioned on a top surface of the handheld device, wherein at least one of the input buttons is for selectively navigating through predetermined sections of a form, and the set of input buttons including a button for selecting a predetermined section of a form and a button for initiating recording,
wherein the at least one of the input buttons for selectively navigating predetermined sections of the text, the button for selecting a predetermined section of a form, and the button for initiating recording are positioned to allow one-handed navigation through the predetermined sections of a form, selection of a predetermined section of a form, and initiation of recording.
2. The handheld device of claim 1, wherein the set of input buttons includes a button for advancing to a next predetermined section of a form and a button for going back to a previous predetermined section of a form.
3. The handheld device of claim 1, wherein the set of input buttons includes a button for selecting a portion of text.
4. The handheld device of claim 1, wherein the set of input buttons includes a button for reviewing back through an audio file to a previously recorded portion of the file, a button for advancing through an audio file to a previously recorded portion of the file, and a button for playing a portion of an audio file through an integrated speaker on the handheld device.
5. The handheld device of claim 4, wherein the button for playing a portion of an audio file is also a stop button, wherein when any function of the handheld device is active, pressing the stop button deactivates the handheld device.
6. The handheld device of claim 1, wherein the set of input buttons includes a button for dictation, wherein when the button for dictation is pressed, the microphone element is activated to receive audio input in the form of speech, the audio input is digitized and transmitted to the speech recognition engine, and the speech recognition engine translates the speech into text.
7. The handheld device of claim 1, further comprising a pointing device on the top surface of the handheld device and at least one button associated with the pointing device.
8. The handheld device of claim 7, wherein the pointing device is a microjoystick.
9. The handheld device of claim 7, wherein the at least one button associated with the pointing device is on the bottom surface of the handheld device.
10. The handheld device of claim 1, wherein the set of input buttons includes a button for signing a form, wherein when the button for signing a form is pressed, the form is modified to include an indicator that the form has been signed.
11. The handheld device of claim 1, wherein the set of input buttons includes a button for coding.
12. A handheld device for receiving audio input in the form of speech, the audio input recorded in a recording medium and processed by a speech recognition engine, thereby generating text, the handheld device comprising:
a USB hub for receiving and transmitting signals through a USB interface to a USB root hub of a computer,
a power switch for switching power drawn from the USB interface between a first USB port on the USB hub and a second USB port on the USB hub,
a pointing device connected to the second USB port on the USB hub, and
a USB streaming controller connected to the first USB port on the USB hub, wherein the USB streaming controller receives digital audio signals from an audio codec that converts analog audio signals from a microphone element into the digital audio signals.
13. The handheld device of claim 12, further comprising a preamplifier between the audio codec and the microphone for amplifying the analog audio signals from the microphone before it is digitized by the audio codec.
14. The handheld device of claim 13, further comprising a speaker, wherein digital audio signals from the USB streaming controller are converted into analog audio signals by the audio codec, the analog audio signals are amplified by an amplifier, and the amplified analog audio signals drive the speaker.
15. The handheld device of claim 13, further comprising a plurality of buttons connected to a plurality of switches, wherein when a button is pressed a corresponding switch is closed, and the USB streaming controller detects the closed switch.
16. The handheld device of claim 15, further comprising at least one LED connected to the USB streaming controller, wherein when one of the plurality of the buttons is pressed and the corresponding switch is closed, a recording function of the handheld device is activated and the at least one LED is lit.
17. The handheld device of claim 12, further comprising a data storage device.
18. The handheld device of claim 12, further comprising a fingerprint security device.
19. The handheld device of claim 17, further comprising a fingerprint security device.
20. A handheld device for receiving audio input in the form of speech, the audio input recorded in a recording medium and processed by a speech recognition engine, thereby generating text, the handheld device comprising:
a USB hub for receiving and transmitting signals through a USB interface to a USB root hub of a computer,
a power switch for switching power drawn from the USB interface between a first USB port on the USB hub and a second USB port on the USB hub,
a pointing device connected to the second USB port on the USB hub,
a USB streaming controller connected to the first USB port on the USB hub, wherein the USB streaming controller receives digital audio signals from an audio codec that converts analog audio signals from a microphone element into the digital audio signals,
a barcode scanner for scanning barcodes to identify the subject matter of the audio input.
21. The handheld device of claim 20, further comprising a preamplifier between the audio codec and the microphone for amplifying the analog audio signals from the microphone before it is digitized by the audio codec.
22. The handheld device of claim 21, further comprising a speaker, wherein digital audio signals from the USB streaming controller are converted into analog audio signals by the audio codec, the analog audio signals are amplified by an amplifier, and the amplified analog audio signals drive the speaker.
23. The handheld device of claim 21, further comprising a plurality of buttons connected to a plurality of switches, wherein when a button is pressed a corresponding switch is closed, and the USB streaming controller detects the closed switch.
24. The handheld device of claim 23, further comprising at least one LED connected to the USB streaming controller, wherein when one of the plurality of the buttons is pressed and the corresponding switch is closed, a recording function of the handheld device is activated and the at least one LED is lit.
25. The handheld device of claim 20, further comprising a data storage device.
26. The handheld device of claim 20, further comprising a fingerprint security device.
27. The handheld device of claim 25, further comprising a fingerprint security device.
US10/424,139 2003-04-28 2003-04-28 USB dictation device Expired - Lifetime US7263483B2 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
US10/424,139 US7263483B2 (en) 2003-04-28 2003-04-28 USB dictation device
PCT/US2004/012978 WO2004098158A2 (en) 2003-04-28 2004-04-28 Usb dictation device
CA2523599A CA2523599C (en) 2003-04-28 2004-04-28 Usb dictation device
EP04760411A EP1623410A4 (en) 2003-04-28 2004-04-28 Usb dictation device
US11/830,871 US9100742B2 (en) 2003-04-28 2007-07-31 USB dictation device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US10/424,139 US7263483B2 (en) 2003-04-28 2003-04-28 USB dictation device

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US11/830,871 Division US9100742B2 (en) 2003-04-28 2007-07-31 USB dictation device

Publications (2)

Publication Number Publication Date
US20040215452A1 true US20040215452A1 (en) 2004-10-28
US7263483B2 US7263483B2 (en) 2007-08-28

Family

ID=33299284

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/424,139 Expired - Lifetime US7263483B2 (en) 2003-04-28 2003-04-28 USB dictation device
US11/830,871 Expired - Fee Related US9100742B2 (en) 2003-04-28 2007-07-31 USB dictation device

Family Applications After (1)

Application Number Title Priority Date Filing Date
US11/830,871 Expired - Fee Related US9100742B2 (en) 2003-04-28 2007-07-31 USB dictation device

Country Status (4)

Country Link
US (2) US7263483B2 (en)
EP (1) EP1623410A4 (en)
CA (1) CA2523599C (en)
WO (1) WO2004098158A2 (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070143505A1 (en) * 2005-12-21 2007-06-21 Portal Player, Inc. Passive usb power configuration switching
US20070192519A1 (en) * 2006-02-13 2007-08-16 Benq Corporation Electronic device capable of extending usb port and concealing power line supplying thereto and electronic apparatus having the same
US20080147409A1 (en) * 2006-12-18 2008-06-19 Robert Taormina System, apparatus and method for providing global communications
US20100161857A1 (en) * 2008-12-22 2010-06-24 Solid State System Co., Ltd. Usb audio controller
US20100161856A1 (en) * 2008-12-22 2010-06-24 Solid State System Co., Ltd. Usb audio and mobile audio system using usb audio controller
US20130175348A1 (en) * 2007-03-23 2013-07-11 Ltt, Ltd Method and apparatus for using a limited capacity portable data carrier
US20130282380A1 (en) * 2012-04-20 2013-10-24 Nuance Communications, Inc. Method And System For Facilitating Communications For A User Transaction
US20140365214A1 (en) * 2013-06-11 2014-12-11 Plantronics, Inc. Character Data Entry
CN105359132A (en) * 2013-06-18 2016-02-24 Mtcom公司 Method for generating and retrieving electronic document and recording medium therefor
US9395799B2 (en) 2012-08-09 2016-07-19 Nvidia Corporation Power management techniques for USB interfaces
CN108461991A (en) * 2018-05-25 2018-08-28 山东交通学院 A kind of code editor easy to use

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6934677B2 (en) 2001-12-14 2005-08-23 Microsoft Corporation Quantization matrices based on critical band pattern information for digital audio wherein quantization bands differ from critical bands
US7502743B2 (en) 2002-09-04 2009-03-10 Microsoft Corporation Multi-channel audio encoding and decoding with multi-channel transform selection
US20040254663A1 (en) * 2003-06-12 2004-12-16 Dame Stephen G. Device for providing audio output and related systems and methods
US8155957B1 (en) * 2003-11-21 2012-04-10 Takens Luann C Medical transcription system including automated formatting means and associated method
GB0329572D0 (en) * 2003-12-20 2004-01-28 Astrazeneca Ab Amide derivatives
EP1612659A1 (en) * 2004-06-29 2006-01-04 GMB Tech (Holland) B.V. Sound recording system and method
US20060242346A2 (en) * 2004-10-06 2006-10-26 Cory Vuong Method and aparatus for plug-and-play webserver
US7697827B2 (en) 2005-10-17 2010-04-13 Konicek Jeffrey C User-friendlier interfaces for a camera
US20070084930A1 (en) * 2005-10-18 2007-04-19 Hitoshi Watanabe Barcode reader with a permanent storage
US20100169092A1 (en) * 2008-11-26 2010-07-01 Backes Steven J Voice interface ocx
US8274772B2 (en) * 2008-12-22 2012-09-25 Conexant Systems, Inc. Current detection and limiting method and apparatus
US8965759B2 (en) * 2012-09-01 2015-02-24 Sarah Hershenhorn Digital voice memo transfer and processing
CN110111790B (en) * 2019-05-09 2022-01-28 海信视像科技股份有限公司 Method, device and equipment for processing external equipment access of intelligent equipment

Citations (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4523055A (en) * 1983-11-25 1985-06-11 Pitney Bowes Inc. Voice/text storage and retrieval system
US4776016A (en) * 1985-11-21 1988-10-04 Position Orientation Systems, Inc. Voice control system
US4969180A (en) * 1989-05-18 1990-11-06 I.I.N.V. Computing Ltd. Cordless pen telephone handset
US5197052A (en) * 1988-03-10 1993-03-23 Grundig E.M.V. Personal computer dictation system with voice anad text stored on the same storage medium
US5365434A (en) * 1993-06-04 1994-11-15 Carolyn E. Carlson Book enhancer
US5402518A (en) * 1992-07-22 1995-03-28 Pcvoice, Inc. Sound storage and sound retrieval system having peripheral with hand operable switches
US5668928A (en) * 1995-01-31 1997-09-16 Kor Team International, Inc. Speech recognition system and method with automatic syntax generation
US5721827A (en) * 1996-10-02 1998-02-24 James Logan System for electrically distributing personalized information
US5724106A (en) * 1995-07-17 1998-03-03 Gateway 2000, Inc. Hand held remote control device with trigger button
US5732216A (en) * 1996-10-02 1998-03-24 Internet Angles, Inc. Audio message exchange system
US5754430A (en) * 1994-03-29 1998-05-19 Honda Giken Kogyo Kabushiki Kaisha Car navigation system
US5781643A (en) * 1996-08-16 1998-07-14 Shure Brothers Incorporated Microphone plosive effects reduction techniques
US5812977A (en) * 1996-08-13 1998-09-22 Applied Voice Recognition L.P. Voice control computer interface enabling implementation of common subroutines
US5818800A (en) * 1992-04-06 1998-10-06 Barker; Bruce J. Voice recording device having portable and local modes of operation
US5838313A (en) * 1995-11-20 1998-11-17 Siemens Corporate Research, Inc. Multimedia-based reporting system with recording and playback of dynamic annotation
US5857099A (en) * 1996-09-27 1999-01-05 Allvoice Computing Plc Speech-to-text dictation system with audio message capability
US5875448A (en) * 1996-10-08 1999-02-23 Boys; Donald R. Data stream editing system including a hand-held voice-editing apparatus having a position-finding enunciator
US6001015A (en) * 1995-10-09 1999-12-14 Nintendo Co., Ltd. Operation controlling device and video processing system used therewith
US6023779A (en) * 1996-01-18 2000-02-08 Pocketscience, Inc. Electronic, acoustical tone generating communications system and method
US6049796A (en) * 1997-02-24 2000-04-11 Nokia Mobile Phones Limited Personal digital assistant with real time search capability
US6108415A (en) * 1996-10-17 2000-08-22 Andrea Electronics Corporation Noise cancelling acoustical improvement to a communications device
US6138036A (en) * 1997-03-13 2000-10-24 Oki Telecom, Inc. Wireless telephone with voice data interface mode
US6169911B1 (en) * 1997-09-26 2001-01-02 Sun Microsystems, Inc. Graphical user interface for a portable telephone
US6184804B1 (en) * 1996-01-26 2001-02-06 Orang-Otang Computers, Inc. Key palette
US6215992B1 (en) * 1997-07-29 2001-04-10 Dennis S. Howell Universal dictation input apparatus and method
US6222909B1 (en) * 1997-11-14 2001-04-24 Lucent Technologies Inc. Audio note taking system and method for communication devices
US6289213B1 (en) * 1996-02-14 2001-09-11 International Business Machines Corporation Computers integrated with a cordless telephone
US20020072820A1 (en) * 2000-12-07 2002-06-13 Thackston James D. System and process method for facilitating efficient communication of specifications for parts and assemblies with a mechanism for assigning responsibility
US20020147390A1 (en) * 2000-12-20 2002-10-10 Markis John Emmanuel M.D. Methods and apparatus for acquiring and using bedside medical data
US20030024975A1 (en) * 2001-07-18 2003-02-06 Rajasekharan Ajit V. System and method for authoring and providing information relevant to the physical world
US6614911B1 (en) * 1999-11-19 2003-09-02 Gentex Corporation Microphone assembly having a windscreen of high acoustic resistivity and/or hydrophobic material
US6697841B1 (en) * 1997-06-24 2004-02-24 Dictaphone Corporation Dictation system employing computer-to-computer transmission of voice files controlled by hand microphone

Family Cites Families (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AT390685B (en) * 1988-10-25 1990-06-11 Philips Nv TEXT PROCESSING SYSTEM
JPH04235630A (en) * 1990-07-30 1992-08-24 Ronald L Colier Method and device for conducting programming of computer by audible command
WO1994020899A1 (en) * 1993-03-10 1994-09-15 Barker Bruce Data entry device
WO1996025738A1 (en) * 1995-02-16 1996-08-22 Lehmann Michael H Portable digital dictation device with remote communication
AU713458B2 (en) * 1996-03-08 1999-12-02 Koninklijke Philips Electronics N.V. Dictation and transcription apparatus
US5899976A (en) * 1996-10-31 1999-05-04 Microsoft Corporation Method and system for buffering recognized words during speech recognition
US5884258A (en) * 1996-10-31 1999-03-16 Microsoft Corporation Method and system for editing phrases during continuous speech recognition
US5874939A (en) * 1996-12-10 1999-02-23 Motorola, Inc. Keyboard apparatus and method with voice recognition
US6529450B1 (en) * 1998-08-03 2003-03-04 Dictaphone Corporation Hand microphone interfaced to game controller port of personal computer
US20020095290A1 (en) * 1999-02-05 2002-07-18 Jonathan Kahn Speech recognition program mapping tool to align an audio file to verbatim text
US6516340B2 (en) * 1999-07-08 2003-02-04 Central Coast Patent Agency, Inc. Method and apparatus for creating and executing internet based lectures using public domain web page
WO2001004875A1 (en) * 1999-07-08 2001-01-18 Koninklijke Philips Electronics N.V. Speech recognition device with transfer means
AU777440B2 (en) * 1999-08-09 2004-10-14 Wake Forest University A method and computer-implemented procedure for creating electronic, multimedia reports
JP4320487B2 (en) * 1999-09-03 2009-08-26 ソニー株式会社 Information processing apparatus and method, and program storage medium
US7047191B2 (en) * 2000-03-06 2006-05-16 Rochester Institute Of Technology Method and system for providing automated captioning for AV signals
US7047192B2 (en) * 2000-06-28 2006-05-16 Poirier Darrell A Simultaneous multi-user real-time speech recognition system
GB2365188B (en) * 2000-07-20 2004-10-20 Canon Kk Method for entering characters
DE10060587A1 (en) * 2000-12-06 2002-06-13 Philips Corp Intellectual Pty Automatic control of actions and operations during lectures and recitals, involves comparing lecture or recital text with stored words or word combinations
US20020077830A1 (en) * 2000-12-19 2002-06-20 Nokia Corporation Method for activating context sensitive speech recognition in a terminal
US6999591B2 (en) * 2001-02-27 2006-02-14 International Business Machines Corporation Audio device characterization for accurate predictable volume control
US6931263B1 (en) * 2001-03-14 2005-08-16 Matsushita Mobile Communications Development Corporation Of U.S.A. Voice activated text strings for electronic devices
WO2002077975A1 (en) * 2001-03-27 2002-10-03 Koninklijke Philips Electronics N.V. Method to select and send text messages with a mobile
WO2002080144A1 (en) * 2001-03-29 2002-10-10 Koninklijke Philips Electronics N.V. Text editing for recognized speech during synchronous playback
JP5093966B2 (en) * 2001-03-29 2012-12-12 ニュアンス コミュニケーションズ オーストリア ゲーエムベーハー Alignment of voice cursor and text cursor during editing
US6834264B2 (en) * 2001-03-29 2004-12-21 Provox Technologies Corporation Method and apparatus for voice dictation and document production
US7318031B2 (en) * 2001-05-09 2008-01-08 International Business Machines Corporation Apparatus, system and method for providing speech recognition assist in call handover
EP1262954A1 (en) * 2001-05-30 2002-12-04 Telefonaktiebolaget L M Ericsson (Publ) Method and apparatus for verbal entry of digits or commands
ATE407411T1 (en) * 2001-07-05 2008-09-15 Koninkl Philips Electronics Nv METHOD FOR PROVIDING ACCOUNT INFORMATION AND SYSTEM FOR WRITING DICTATE TEXT
WO2004023455A2 (en) * 2002-09-06 2004-03-18 Voice Signal Technologies, Inc. Methods, systems, and programming for performing speech recognition
JP4241376B2 (en) * 2001-09-17 2009-03-18 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ Correction of text recognized by speech recognition through comparison of speech sequences in recognized text with speech transcription of manually entered correction words
US6708148B2 (en) * 2001-10-12 2004-03-16 Koninklijke Philips Electronics N.V. Correction device to mark parts of a recognized text
US20030083903A1 (en) * 2001-10-30 2003-05-01 Myers Gene E. Method and apparatus for contemporaneous billing and documenting with rendered services
US20030146942A1 (en) * 2002-02-07 2003-08-07 Decode Genetics Ehf. Medical advice expert
JP3615195B2 (en) * 2002-03-19 2005-01-26 株式会社東芝 Content recording / playback apparatus and content editing method
AU2002314439A1 (en) * 2002-07-02 2004-01-23 Nokia Corporation Method and communication device for handling data records by speech recognition
US7444285B2 (en) * 2002-12-06 2008-10-28 3M Innovative Properties Company Method and system for sequential insertion of speech recognition results to facilitate deferred transcription services

Patent Citations (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4523055A (en) * 1983-11-25 1985-06-11 Pitney Bowes Inc. Voice/text storage and retrieval system
US4776016A (en) * 1985-11-21 1988-10-04 Position Orientation Systems, Inc. Voice control system
US5197052A (en) * 1988-03-10 1993-03-23 Grundig E.M.V. Personal computer dictation system with voice anad text stored on the same storage medium
US4969180A (en) * 1989-05-18 1990-11-06 I.I.N.V. Computing Ltd. Cordless pen telephone handset
US5818800A (en) * 1992-04-06 1998-10-06 Barker; Bruce J. Voice recording device having portable and local modes of operation
US5402518A (en) * 1992-07-22 1995-03-28 Pcvoice, Inc. Sound storage and sound retrieval system having peripheral with hand operable switches
US5365434A (en) * 1993-06-04 1994-11-15 Carolyn E. Carlson Book enhancer
US5754430A (en) * 1994-03-29 1998-05-19 Honda Giken Kogyo Kabushiki Kaisha Car navigation system
US5668928A (en) * 1995-01-31 1997-09-16 Kor Team International, Inc. Speech recognition system and method with automatic syntax generation
US5724106A (en) * 1995-07-17 1998-03-03 Gateway 2000, Inc. Hand held remote control device with trigger button
US6001015A (en) * 1995-10-09 1999-12-14 Nintendo Co., Ltd. Operation controlling device and video processing system used therewith
US5838313A (en) * 1995-11-20 1998-11-17 Siemens Corporate Research, Inc. Multimedia-based reporting system with recording and playback of dynamic annotation
US6023779A (en) * 1996-01-18 2000-02-08 Pocketscience, Inc. Electronic, acoustical tone generating communications system and method
US6184804B1 (en) * 1996-01-26 2001-02-06 Orang-Otang Computers, Inc. Key palette
US6289213B1 (en) * 1996-02-14 2001-09-11 International Business Machines Corporation Computers integrated with a cordless telephone
US5812977A (en) * 1996-08-13 1998-09-22 Applied Voice Recognition L.P. Voice control computer interface enabling implementation of common subroutines
US5781643A (en) * 1996-08-16 1998-07-14 Shure Brothers Incorporated Microphone plosive effects reduction techniques
US5857099A (en) * 1996-09-27 1999-01-05 Allvoice Computing Plc Speech-to-text dictation system with audio message capability
US5732216A (en) * 1996-10-02 1998-03-24 Internet Angles, Inc. Audio message exchange system
US5721827A (en) * 1996-10-02 1998-02-24 James Logan System for electrically distributing personalized information
US5875448A (en) * 1996-10-08 1999-02-23 Boys; Donald R. Data stream editing system including a hand-held voice-editing apparatus having a position-finding enunciator
US6108415A (en) * 1996-10-17 2000-08-22 Andrea Electronics Corporation Noise cancelling acoustical improvement to a communications device
US6049796A (en) * 1997-02-24 2000-04-11 Nokia Mobile Phones Limited Personal digital assistant with real time search capability
US6138036A (en) * 1997-03-13 2000-10-24 Oki Telecom, Inc. Wireless telephone with voice data interface mode
US6697841B1 (en) * 1997-06-24 2004-02-24 Dictaphone Corporation Dictation system employing computer-to-computer transmission of voice files controlled by hand microphone
US6215992B1 (en) * 1997-07-29 2001-04-10 Dennis S. Howell Universal dictation input apparatus and method
US6169911B1 (en) * 1997-09-26 2001-01-02 Sun Microsystems, Inc. Graphical user interface for a portable telephone
US6222909B1 (en) * 1997-11-14 2001-04-24 Lucent Technologies Inc. Audio note taking system and method for communication devices
US6614911B1 (en) * 1999-11-19 2003-09-02 Gentex Corporation Microphone assembly having a windscreen of high acoustic resistivity and/or hydrophobic material
US20020072820A1 (en) * 2000-12-07 2002-06-13 Thackston James D. System and process method for facilitating efficient communication of specifications for parts and assemblies with a mechanism for assigning responsibility
US20020147390A1 (en) * 2000-12-20 2002-10-10 Markis John Emmanuel M.D. Methods and apparatus for acquiring and using bedside medical data
US20030024975A1 (en) * 2001-07-18 2003-02-06 Rajasekharan Ajit V. System and method for authoring and providing information relevant to the physical world

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7698490B2 (en) * 2005-12-21 2010-04-13 Nvidia Corporation Passive USB power configuration switching
US20070143505A1 (en) * 2005-12-21 2007-06-21 Portal Player, Inc. Passive usb power configuration switching
US20070192519A1 (en) * 2006-02-13 2007-08-16 Benq Corporation Electronic device capable of extending usb port and concealing power line supplying thereto and electronic apparatus having the same
US20080147409A1 (en) * 2006-12-18 2008-06-19 Robert Taormina System, apparatus and method for providing global communications
US9317792B2 (en) * 2007-03-23 2016-04-19 Ltt, Ltd Method and apparatus for using a limited capacity portable data carrier
US20130175348A1 (en) * 2007-03-23 2013-07-11 Ltt, Ltd Method and apparatus for using a limited capacity portable data carrier
US20100161857A1 (en) * 2008-12-22 2010-06-24 Solid State System Co., Ltd. Usb audio controller
US20100161856A1 (en) * 2008-12-22 2010-06-24 Solid State System Co., Ltd. Usb audio and mobile audio system using usb audio controller
US7962668B2 (en) * 2008-12-22 2011-06-14 Solid State System Co., Ltd. USB audio controller
US20130282380A1 (en) * 2012-04-20 2013-10-24 Nuance Communications, Inc. Method And System For Facilitating Communications For A User Transaction
US9037472B2 (en) * 2012-04-20 2015-05-19 Nuance Communications, Inc. Method and system for facilitating communications for a user transaction
CN103376992A (en) * 2012-04-20 2013-10-30 纽安斯通讯公司 Method and system for facilitating communications for a user transaction
US9395799B2 (en) 2012-08-09 2016-07-19 Nvidia Corporation Power management techniques for USB interfaces
US20140365214A1 (en) * 2013-06-11 2014-12-11 Plantronics, Inc. Character Data Entry
US9449600B2 (en) * 2013-06-11 2016-09-20 Plantronics, Inc. Character data entry
CN105359132A (en) * 2013-06-18 2016-02-24 Mtcom公司 Method for generating and retrieving electronic document and recording medium therefor
CN108461991A (en) * 2018-05-25 2018-08-28 山东交通学院 A kind of code editor easy to use

Also Published As

Publication number Publication date
US9100742B2 (en) 2015-08-04
WO2004098158A3 (en) 2005-12-15
EP1623410A2 (en) 2006-02-08
WO2004098158A2 (en) 2004-11-11
CA2523599A1 (en) 2004-11-11
US20080015857A1 (en) 2008-01-17
US7263483B2 (en) 2007-08-28
CA2523599C (en) 2016-03-22
EP1623410A4 (en) 2009-02-18

Similar Documents

Publication Publication Date Title
US9100742B2 (en) USB dictation device
US6353809B2 (en) Speech recognition with text generation from portions of voice data preselected by manual-input commands
US5477511A (en) Portable documentation system
US6047257A (en) Identification of medical images through speech recognition
US6415258B1 (en) Background audio recovery system
US5526407A (en) Method and apparatus for managing information
US6571211B1 (en) Voice file header data in portable digital audio recorder
US20050114132A1 (en) Voice interactive method and system
US20060195322A1 (en) System and method for detecting and storing important information
US20020095290A1 (en) Speech recognition program mapping tool to align an audio file to verbatim text
PL182225B1 (en) Method of and apparatus for transmitting voice-operated data processing system
US20020065662A1 (en) Voice recognition peripheral device
EP1387349A3 (en) Voice recognition/response system, voice recognition/response program and recording medium for same
JPH10272106A (en) Discrimination of medical treatment image by voice recognition
US20070014540A1 (en) Audio recording apparatus, audio recording method and audio recording program
JPH06131108A (en) Information input device
US8280734B2 (en) Systems and arrangements for titling audio recordings comprising a lingual translation of the title
KR950014504B1 (en) Portable computer device for audible processing of electronic documents
EP1830693A1 (en) Medical media capture system and method
US20010011219A1 (en) Voice recording and reproducing apparatus, information processing apparatus, and recording medium having recorded an information processing program
JPH0883092A (en) Information inputting device and method therefor
JPH1011520A (en) Medical data recorder
JP2000075893A (en) Voice recognition device
JP2017194794A (en) Voice texting system, information acquisition instrument and dictation device
JPH11212590A (en) Voice processor, recording medium with voice recognition program recorded, and recording medium with processing program recorded

Legal Events

Date Code Title Description
AS Assignment

Owner name: DICTAPHONE CORPORATION, CONNECTICUT

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PEARAH, DAVID E.;CHEN, FRANCIS;REEL/FRAME:014329/0180

Effective date: 20030515

AS Assignment

Owner name: USB AG, STAMFORD BRANCH,CONNECTICUT

Free format text: SECURITY AGREEMENT;ASSIGNOR:NUANCE COMMUNICATIONS, INC.;REEL/FRAME:017435/0199

Effective date: 20060331

Owner name: USB AG, STAMFORD BRANCH, CONNECTICUT

Free format text: SECURITY AGREEMENT;ASSIGNOR:NUANCE COMMUNICATIONS, INC.;REEL/FRAME:017435/0199

Effective date: 20060331

AS Assignment

Owner name: USB AG. STAMFORD BRANCH,CONNECTICUT

Free format text: SECURITY AGREEMENT;ASSIGNOR:NUANCE COMMUNICATIONS, INC.;REEL/FRAME:018160/0909

Effective date: 20060331

Owner name: USB AG. STAMFORD BRANCH, CONNECTICUT

Free format text: SECURITY AGREEMENT;ASSIGNOR:NUANCE COMMUNICATIONS, INC.;REEL/FRAME:018160/0909

Effective date: 20060331

STCF Information on status: patent grant

Free format text: PATENTED CASE

FPAY Fee payment

Year of fee payment: 4

AS Assignment

Owner name: NUANCE COMMUNICATIONS, INC., MASSACHUSETTS

Free format text: MERGER;ASSIGNOR:DICTAPHONE CORPORATION;REEL/FRAME:028952/0397

Effective date: 20060207

AS Assignment

Owner name: NUANCE COMMUNICATIONS, INC., MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DICTAPHONE CORPORATION;REEL/FRAME:029596/0836

Effective date: 20121211

FPAY Fee payment

Year of fee payment: 8

AS Assignment

Owner name: TELELOGUE, INC., A DELAWARE CORPORATION, AS GRANTO

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: SPEECHWORKS INTERNATIONAL, INC., A DELAWARE CORPOR

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: DICTAPHONE CORPORATION, A DELAWARE CORPORATION, AS

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: NORTHROP GRUMMAN CORPORATION, A DELAWARE CORPORATI

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: INSTITIT KATALIZA IMENI G.K. BORESKOVA SIBIRSKOGO

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: ART ADVANCED RECOGNITION TECHNOLOGIES, INC., A DEL

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: NOKIA CORPORATION, AS GRANTOR, FINLAND

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: DSP, INC., D/B/A DIAMOND EQUIPMENT, A MAINE CORPOR

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: SPEECHWORKS INTERNATIONAL, INC., A DELAWARE CORPOR

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: SCANSOFT, INC., A DELAWARE CORPORATION, AS GRANTOR

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: TELELOGUE, INC., A DELAWARE CORPORATION, AS GRANTO

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: SCANSOFT, INC., A DELAWARE CORPORATION, AS GRANTOR

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: DICTAPHONE CORPORATION, A DELAWARE CORPORATION, AS

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: NUANCE COMMUNICATIONS, INC., AS GRANTOR, MASSACHUS

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: ART ADVANCED RECOGNITION TECHNOLOGIES, INC., A DEL

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: NUANCE COMMUNICATIONS, INC., AS GRANTOR, MASSACHUS

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: MITSUBISH DENKI KABUSHIKI KAISHA, AS GRANTOR, JAPA

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: STRYKER LEIBINGER GMBH & CO., KG, AS GRANTOR, GERM

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: DSP, INC., D/B/A DIAMOND EQUIPMENT, A MAINE CORPOR

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: HUMAN CAPITAL RESOURCES, INC., A DELAWARE CORPORAT

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FEPP Fee payment procedure

Free format text: 11.5 YR SURCHARGE- LATE PMT W/IN 6 MO, LARGE ENTITY (ORIGINAL EVENT CODE: M1556); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 12