WO2001037095A1 - Method and system for intercepting an application program interface - Google Patents

Method and system for intercepting an application program interface Download PDF

Info

Publication number
WO2001037095A1
WO2001037095A1 PCT/US2000/031032 US0031032W WO0137095A1 WO 2001037095 A1 WO2001037095 A1 WO 2001037095A1 US 0031032 W US0031032 W US 0031032W WO 0137095 A1 WO0137095 A1 WO 0137095A1
Authority
WO
WIPO (PCT)
Prior art keywords
address
call
intercepted
library
routine
Prior art date
Application number
PCT/US2000/031032
Other languages
French (fr)
Inventor
Yona Hollander
Ophir Rahman
Oded Horvitz
Original Assignee
Clicknet Software, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from IL13291699A external-priority patent/IL132916A/en
Application filed by Clicknet Software, Inc. filed Critical Clicknet Software, Inc.
Priority to AU15986/01A priority Critical patent/AU1598601A/en
Priority to JP2001539121A priority patent/JP2003515219A/en
Priority to EP00978530A priority patent/EP1236114A1/en
Publication of WO2001037095A1 publication Critical patent/WO2001037095A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/52Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity ; Preventing unwanted data erasure; Buffer overflow
    • G06F21/53Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity ; Preventing unwanted data erasure; Buffer overflow by executing in a restricted environment, e.g. sandbox or secure virtual machine
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/14Protection against unauthorised use of memory or access to memory
    • G06F12/1416Protection against unauthorised use of memory or access to memory by checking the object accessibility, e.g. type of access defined by the memory independently of subject rights
    • G06F12/1425Protection against unauthorised use of memory or access to memory by checking the object accessibility, e.g. type of access defined by the memory independently of subject rights the protection being physical, e.g. cell, word, block
    • G06F12/1441Protection against unauthorised use of memory or access to memory by checking the object accessibility, e.g. type of access defined by the memory independently of subject rights the protection being physical, e.g. cell, word, block for a range
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/448Execution paradigms, e.g. implementations of programming paradigms
    • G06F9/4482Procedural
    • G06F9/4484Executing subprograms
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/54Indexing scheme relating to G06F9/54
    • G06F2209/542Intercept

Definitions

  • the present invention relates generally to a method for
  • the present invention relates to a computer system. More specifically, the present invention relates to a
  • Procedures or functions are computer programs. A procedure
  • a stack is a contiguous block of memory containing data. Its
  • CPU Central Processing Unit
  • the stack consists of logical stack frames or Procedure
  • the stack frame itself contains
  • the return address is the instruction pointer of the
  • One such objective is inserting an
  • attack code in the form of an executable binary code native to the
  • Another such objective is to change the return
  • Another strategy is to detect buffer
  • said operating system including a kernel space and a process
  • process space including a user application running in
  • process space said user application operative to intercept system calls
  • said method comprising the step of examining said intercepted system
  • the present invention a method of secure function execution within a
  • said operating system including a kernel space and a process
  • process space including a user application running in
  • said user application operative to intercept library calls
  • said method comprising the step of examining said intercepted library
  • said operating system including a kernel space and a process
  • process space including a user application running in
  • said user application operative to intercept library calls
  • said method comprising the step of examining said intercepted library
  • said operating system including a kernel space and a process
  • process space including a user application running in
  • said operating system including a kernel space and a process
  • process space including a user application running in
  • said process memory device further comprises the step of determining
  • Fig. 1 is a block diagram of the Secure Function Execution
  • system environment generally referenced to as system 100;
  • Fig. 2 is a high-level flow diagram of the Secure Function
  • Fig. 3 is a high-level flow diagram of the operation of the
  • Fig. 4 is a high-level diagram of Secure Function Execution
  • Fig. 5 is a high-level flow diagram of the operation of the /37095
  • Fig. 6 is a high-level flow diagram of the operation of the
  • Fig. 7 is a high-level flow diagram of the Calling Address
  • the present invention is related to Israel Patent Application
  • system 100 of Fig. 1 may comprise of four
  • Secure Function Execution Server 116 is the operational center of the Secure Function Execution
  • Secure Function Execution Server 116 incorporates the
  • API Interception Module 134 140, 146 and the like are
  • Interception Module 134, 140, 146 and the like consist of
  • API routine 132, 138, 144 and the like are passive
  • API routines 132, 138, 144 and the like are
  • FIG. 2 there is provided a high-level flow
  • Server 116 initializes the
  • step 152 run-time operation in step 152 by constantly monitoring system calls
  • SFE Server is also constantly constantly
  • step 156 SFE Server responds appropriately to the
  • First SFE Server 116 loads System Call Interception
  • step 186 For the list of active processes 118, 120, 122 and the like (step 186).
  • Server 116 creates a list of valid address ranges for each active
  • DLL Dynamic Link Library
  • Dynamic Link Library is a set of callable subroutines
  • SFE Server will insert API Interception Module 134,
  • FIG. 4 is a high-level flow diagram of
  • step 160 determines in step 160
  • Server determines whether said system call is valid by comparing said
  • the SFE Server 116 may terminate the illegal call
  • SFE Server 116 may notify a user
  • SFE Server 116 may perform another or other series of
  • decision in step 162 is negative SFE Server optionally performs any
  • step 166 If and when it
  • FIG. 5 is a high-level flow diagram of the
  • SFE Server 116 determines in step 172 if the
  • SFE Server determines whether
  • said library call is valid by comparing said library call originating
  • detected SFE Server 116 optionally terminates the illegal library
  • SFE Server 116 notifies a user
  • Server 116 performs any other user predetermined or instructed action
  • step 174 If the decision in step 174 is affirmative than SFE Server 116
  • process 118 is now operative to intercept calls made to said library calls
  • step 172 decision in step 172 is negative SFE Server 116 determines if the
  • the Calling Address Validation Routine module may operate in
  • Pre-Entry routine may be activated when an API 132 or the
  • Calling Address Validation Routine module is executing a set of
  • Caller Routine also includes caller Application
  • the stack frame is a dynamic area of the process
  • stack segment is a dynamic area of memory belonging to a process.
  • step 192 the caller Routine calling address is calculated (step 192) and
  • step 194 it is determined whether
  • Fig. 7 is a high-level flow
  • Routine return address is significantly faster and more accurate.
  • Such determination is accomplished by comparing said caller
  • Pre-Entry routine or the like notifies SFE Server 116 or the like about
  • step 210 and step 212 the result of the examination.

Abstract

In a computer system running an operating system platform, having an operating system including a kernel space and a process space, said process space including a user application running in process space, said user application operative to intercept system calls, a method of secure function execution, said method comprising the step of examining said intercepted system call (191, 192) validity by comparing said intercepted system call originating address with range of process valid addresses (194) associated with said process from which said intercepted system call originated (191, 192) and providing notification (198, 200) as to the validity (196) of said intercepted system call (191, 192) or terminating said intercepted system call.

Description

METHOD AND δYSTEM FOR INTERCEPTING AN APPLICATION PROGRAM INTERFACE
FIELD OF THE INVENTION
The present invention relates generally to a method for
detecting and preventing unauthorized or illegal access attempts within
a computer system. More specifically, the present invention relates to a
method for detecting and preventing attempts to exploit the buffer
overflow-related weakness within a computer system.
BACKGROUND OF THE INVENTION
This application is related to Israel Patent Application Number
"METHOD AND SYSTEM FOR INTERCEPTING A
APPLICATION PROGRAM INTERFACE" filed 14 November 1999.
Modern computers are designed with the requirements of
high-level languages in mind. The most essential technique for
structuring computer programs introduced by high-level languages, is
the procedure or the function.
Procedures or functions are computer programs. A procedure
call or a function call is a high-level abstraction that alters the flow of
the calling program execution. In contrast with the more traditional
"jump" or "goto" instructions, which also alter the flow of execution, a
procedure or a function, after the execution, of its own code, returns
control to the instruction immediately following the call. To implement 7095
procedure or function calls in the manner described, a memory device
called a stack is utilized.
A stack is a contiguous block of memory containing data. Its
size is dynamically adjusted by the operating system routines at run
time. The data is inserted to and removed from the stack by Central
Processing Unit (CPU) utilizing Assembler language instructions such
as "push" or "pop".
The stack consists of logical stack frames or Procedure
Activation Records that are inserted into the stack when a function is
called and removed from the stack when the said function returns
control to the calling program. The stack frame itself contains
parameters to the called function, local variables, pointers to recover
the previous stack frame, and the return address of the calling
computer program. The return address is the instruction pointer of the
calling program at the time of the function call.
Induced buffer overflow or buffer overflow attack is known in
the art. Buffer overflow attacks exploit the lack of bounds checking on
the size of input being stored in a buffer array. Arrays are predefined
allocated memory devices within a computer system. By writing data
intentionally past the end of an allocated array, an attacker can make
arbitrary changes to data stored adjacent to the said array. The most common data structure to be corrupted in this fashion is the stack.
Therefore this type of attack is also known as stack smashing.
The prevalent form of buffer overflow exploitation is to attack
buffers allocated on the stack. Such attacks attempt to achieve two
mutually dependent objectives. One such objective is inserting an
attack code in the form of an executable binary code native to the
attacked machine. Another such objective is to change the return
address to point to the attacker's supplied code now residing within said
stack memory. Such attacker's supplied code may be utilized to gain
enhanced privileges over said computer system.
The programs that are attacked using this technique are
usually high privilege utilities or daemons that run under the user-id root
to perform essential services. The effect of a successful buffer overflow
attack is to provide the attacker non-authorized root privileges. Gaining
root privileges within a computer system allows non-authorized users
access privileged resources.
As the maximum length of the overflowing data string can be
only the current depth of the stack, the inserted attack code should be
short in terms of code length. Writing data outside the stack limit will
result in an exception condition that will prevent the attack code to
execute. Therefore, the buffer overflow attacker will be forced to write 7095
short code and will have to use high-level System calls or Library calls.
Such calls will later be utilized to gain non-authorized enhanced
privileges to access privileged resources.
Several strategies, which attempt to resolve the buffer
overflow weakness, are known in the art. One such strategy is to
design a compiler designed to prohibit a computer program from writing
past a stack segment array. Another strategy is to detect buffer
overflow vulnerable programs off line and alert the user to the
possibility that the system privileges may be compromised.
Another known strategy is using a repair program. The repair
program can repair or fix those vulnerable programs that can be used
to exploit the buffer overflow weakness.
None of the above provide a method and apparatus for
prevention of buffer overflow through controlled execution of system or
other calls within a computer system.
SUMMARY OF THE PRESENT INVENTION
Thus, there is a long felt need to provide a for detecting and
preventing unauthorized or illegal access attempts within a computer
system. More specifically, a method for detecting and preventing
attempts to exploit the buffer overflow-related weakness within a
computer system by validating system or other calls made within a
computer system.
It is therefore the object of this invention to provide a method
for preventing induced buffer overflow attack by preventing execution of
high-level System calls, Library calls, Application Program Interface call
and the like when such calls are illegally made.
It is therefore another object of the present invention to
provide a method for preventing induced buffer overflow attack by
preventing execution of high-level System calls, Library calls,
Application Program Interface call and the like when such calls are
made from unauthorized areas within a computer system.
It is yet a further object of the present invention to provide a
method for preventing induced buffer overflow attack by preventing
execution of high-level System calls, Library calls, Application Program
Interface call when such calls are made from outside the user process
associated with said called system or other call. 37095
It is therefore provided in accordance with a preferred
embodiment of the present invention a method of secure function
execution within a computer system running an operating system
platform, said operating system including a kernel space and a process
space, said process space including a user application running in
process space, said user application operative to intercept system calls,
said method comprising the step of examining said intercepted system
call validity by comparing said intercepted system call originating
address with range of process valid addresses associated with said
process from which said intercepted system call originated and
providing notification as to the validity of said intercepted system call, or
terminating said intercepted system call.
It is further provided in accordance with a preferred embodiment of
the present invention a method of secure function execution within a
computer system running an operating system platform, said operating
system including a kernel space and a process space, said process
space including a user application running in process space, said user
application operative to intercept system calls, said method comprising
the step of examining said intercepted system call validity by comparing
said intercepted system call originating address with range of process
valid addresses associated with said process from which said intercepted system call originated and responsive to process creation
inserting application program interface interception module into said
created process and responsive to process creation updating process
valid addresses table or responsive to process termination updating
process valid addresses table.
In accordance with yet another preferred embodiment of the
present invention there is provided a method of secure function
execution within a computer system running an operating system
platform, said operating system including a kernel space and a process
space, said process space including a user application running in
process space, said user application operative to intercept library calls,
said method comprising the step of examining said intercepted library
call validity by comparing said intercepted library call originating
address with range of process valid addresses associated with said
process from which said intercepted library call originated and providing
notification as to the validity of ..said intercepted library call or
terminating said intercepted library call.
In accordance with yet another preferred embodiment of the
present invention there is provided a method of secure function 37095
execution within a computer system running an operating system
platform, said operating system including a kernel space and a process
space, said process space including a user application running in
process space, said user application operative to intercept library calls,
said method comprising the step of examining said intercepted library
call validity by comparing said intercepted library call originating
address with range of process valid addresses associated with said
process from which said intercepted library call originated and
responsive to system call loading dynamic link library hooking and
patching library routines associated with said dynamic link library and
responsive to system call unloading dynamic link library updating
process valid addresses table.
In accordance with another preferred embodiment of the
present invention there is provided a method of secure function
execution within a computer system running an operating system
platform, said operating system including a kernel space and a process
space, said process space including a user application running in
process space, said user application operative to system and function
calls, said system or function call intercepted, said method comprising
the steps of receiving caller routine return address from said process memory device, determining whether caller routine address is valid by
comparing said caller address routine with process valid address table
and providing notification as to the validity of said caller routine return
address or performing user predetermined acts associated with said
validity of caller routine address. The same method further comprising
the step of determining said caller routine calling address by
determining the address preceding said caller routine address.
In accordance with yet another preferred embodiment of the
present invention there is provided a method of secure function
execution within a computer system running an operating system
platform, said operating system including a kernel space and a process
space, said process space including a user application running in
process space, said user application operative to system and function
calls, said system or function call intercepted, said method comprising
the steps of receiving caller routine return address from said process
memory device, determining whether caller routine address is valid by
comparing said caller address routine with associated process stack
address area and providing notification as to the validity of said caller
routine return address or performing user predetermined acts
associated with said validity of caller routine address. The same 7095
method wherein the step of receiving caller routine return address from
said process memory device further comprises the step of determining
said caller routine calling address by determining the address
preceding said caller routine address.
37095
BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated in and
constitutes a part of the specification, illustrate preferred embodiments
of the invention and, together with the description, serve to explain the
principles of the invention:
Fig. 1 is a block diagram of the Secure Function Execution
System environment generally referenced to as system 100;
Fig. 2 is a high-level flow diagram of the Secure Function
Execution Server 116 operation;
Fig. 3 is a high-level flow diagram of the operation of the
Secure Function Execution Server initialization module referred to in
Fig. 2;
Fig. 4 is a high-level diagram of Secure Function Execution
Server or the like response to an intercepted system call referred to in
Fig. 2;
Fig. 5 is a high-level flow diagram of the operation of the /37095
Secure Function Execution Server and the like library call response
module referred to in Fig. 2;
Fig. 6 is a high-level flow diagram of the operation of the
Calling Address Validation Routine module;
Fig. 7 is a high-level flow diagram of the Calling Address
Validation Routine module relating to an another embodiment of the
present invention.
7095
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
The present invention overcomes the disadvantages of the
prior art by providing a novel method, which detects if an attempt to
exploit the buffer overflow weakness is occurring by validating use of
system or other calls within a computerized system.
Reference is now made to Fig. 1 , there is provided a
schematic illustration of the system environment wherein the Secure
Function Execution System is operating, generally referred to as
system 100, in accordance with a preferred embodiment of the present
invention.
The present invention is related to Israel Patent Application
Number XXXXXXX. "METHOD AND SYSTEM FOR INTERCEPTING A
APPLICATION PROGRAM INTERFACE" filed 14 November 1999.
As further described in detail in Israel Patent Application
Number XXXXXXX, system 100 of Fig. 1 may comprise of four
components of the Secure Function Execution System;
a) Secure Function Execution Server 116 is an active
component. Secure Function Execution Server 116 is the operational center of the Secure Function Execution
System 100. Secure Function Execution Server 116
loads and controls System Call Interception Component
124, loads and controls API Interception Module 134,
140, and 146, responds to diverse system and library
calls and acts as an interface towards the user. The
Secure Function Execution Server 116 is loaded into the
user space memory device 112 of a computer system.
Secure Function Execution Server 116 incorporates the
API Interception Control Server operations that were
described in detail in Israel Patent Application No.
XXXXXXX.
b) API Interception Module 134, 140, 146 and the like are
active components. API Interception Module 134, 140,
146 and the like operations are described in detail in
Israel Patent Application Number XXXXXXX. API
Interception Module 134, 140, 146 and the like consist of
Dispatch Routine, Depatch Routine, Hook and Patch
Routine, Pre-Entry Routine, and Post-Entry Routine. The
operations of the said routines are also described in
detail in Israel Patent Application No. XXXXXXX. 095
c) System Call Interception Component 124 is an active
component. System Call Interception Component 124
operation is described in detail in Israel Patent
Application No. XXXXXXX.
d) API routine 132, 138, 144 and the like are passive
components. API routines 132, 138, 144 and the like are
potential objects upon which Secure Function Execution
System 100 might operate. API routines 132, 138, 144
and the like are loaded into process address space
memory device 118, 120, 122 and the like in user
memory device 112 of system 100.
System 100 previously described in Israel Patent Application
number XXXXXXX serve as model for a Secure Function Execution
System in which the present invention is operative. It will be
appreciated by those skilled in the art that the present invention may
operate under various similar systems and that the system shown
herein is an example to further illustrate the working of the present
invention.
Referring to Fig. 2 there is provided a high-level flow
diagram of the Secure Function Execution Server 116 operation. Secure Function Execution Server 116 was previously described in
detail in Israel Patent Application Number XXXXXXX. The operation of
said Secure Function Execution Server 116 would now be briefly explained.
Secure Function Execution (SFE as it will be abbreviated
from this point on in the text of this document) Server 116 initializes the
application in step 150. Consequently SFE Server 116 commences its
run-time operation in step 152 by constantly monitoring system calls
made by diverse applications that run in the host operating system
(step 152) and responding appropriately to the said system calls (step
154) as described in detail in Fig. 4. SFE Server is also constantly
monitoring library calls made by diverse application that run in the host
operating system (step 156). SFE Server responds appropriately to the
said library calls (step 158) as described in detail in Fig. 5.
Referring now to Fig. 3 there is provided a high-level flow
diagram of SFE Server 116 start-up.operation referenced as step 150
of Fig. 2. SFE Server 116 start-up operation was previously described
in detail in Israel Patent Application Number XXXXXXX.
First SFE Server 116 loads System Call Interception
Component 124 into kernel space memory device 114 (step 184). After 37095
establishing communication with System Call Interception Component
124 SFE Server 116 queries System Call Interception Component 124
for the list of active processes 118, 120, 122 and the like (step 186).
Using the said list of active processes 118, 120, 122 and the like SFE
Server 116 creates a list of valid address ranges for each active
process 118, 120, 122 and the like. This structure will be referenced
from this point on as Process Valid Address Range in the text of this
document.
Process Valid Address Range List holds the address range
into which the process 118, 120, 122 and the like was loaded to.
Process Valid Address Range also holds all the address ranges into
which diverse Dynamic Link Library (DLL) 130, 136, 142 and the like
were loaded. Dynamic Link Library is a set of callable subroutines
linked as a binary image that can be dynamically loaded by applications
that utilize them.
Finally, SFE Server will insert API Interception Module 134,
140, 146 and the like to all active processes 118, 120, 122 and the like
(step 19) as described in detail in Israel Patent Application No.
XXXXXXX.
SFE Server operation of monitoring system calls in step 152 7095
of Fig. 2 is described in detail in Israel Patent Application No. XXXXXXX.
Turning now to Fig. 4 which is a high-level flow diagram of
SFE Server or the like response to an intercepted system call, referred
to as step 154 of Fig. 2. SFE Server 116 determines in step 160
whether the system call detected is an illegal call or a legal call. SFE
Server determines whether said system call is valid by comparing said
system call originating address with range of Process Valid Address
associated with said process from which said system call originated. If
illegal call was detected the SFE Server 116 may terminate the illegal
function (step 164). Alternatively SFE Server 116 may notify a user
(typically the System Administrator) about the illegal call (step 166).
Alternatively, SFE Server 116 may perform another or other series of
user predetermined actions.
If the system call detected is legal (step 160) SFE Server 116
examines the said system call to determine if it is of the type of process
creation (step 162). If and when it is determined that the system call of
the type of process creation SFE Server 116 inserts API Interception
Module 134 to the newly created process address space 118 (step 168)
and updates Process Valid Address Range List (step 170) by adding 7095
said process address list to Process Valid Address Range. If the
decision in step 162 is negative SFE Server optionally performs any
other user predetermined or instructed action (step 166). If and when it
is determined that the system call of the type of process termination
SFE Server 116 updates Process Valid Address Range List (step 171)
by removing said process valid addresses range from Process Valid Address Range List.
SFE Server operation of monitoring library calls in step 156 of
Fig. 2 is described in detail in US Patent Application No. XXXXXXX.
Turning now to Fig. 5 which is a high-level flow diagram of the
SFE Server and the like response to an intercepted library call referred
to as step 158 of Fig. 2. SFE Server 116 determines in step 172 if the
library call detected is an illegal call. SFE Server determines whether
said library call is valid by comparing said library call originating
address with range of Process Valid Address associated with said
process from which said library call originated. If an illegal call is
detected SFE Server 116 optionally terminates the illegal library
function (step 180). Alternatively, SFE Server 116 notifies a user
(typically the System Administrator) (step 182). Alternatively, SFE 37095
Server 116 performs any other user predetermined or instructed action
(step 182).
If the library call detected is legal (step 172) SFE Server 116
determines if the said library call is of the type of DLL 130 load (step
174). If the decision in step 174 is affirmative than SFE Server 116
hooks and patches the library calls (APIs) 132 existing within said
loaded DLL 130. Such hooking and patching is further described in
detail in Israel Patent Application . After hooking and patching
said API Interception Module 134 already loaded into said associated
process 118 is now operative to intercept calls made to said library calls
132. When determined that the library call is of the type DLL load SFE
Server 116 updates Process Valid Address Range List (step 178) by
adding DLL address rage into Process Valid Address Range List. If the
decision in step 172 is negative SFE Server 116 determines if the
intercepted library call if of the type DLL unload (step 176) by deleting
DLL address range from Process Valid Address Range List. When it is
determined that the library call is of the type of DLL unload SFE Server
updates the Process Valid Address Range List (step 178).
Reference in now made to Fig. 6 that is a high-level flow
diagram of the operation of the Calling Address Validation Routine 37095
module. The Calling Address Validation Routine module may operate in
conjunction with API Interception Module Pre-Entry routine as further
described in detail in Israel Patent Application No. XXXXXXX.
Pre-Entry routine may be activated when an API 132 or the
like of Fig. 1 is intercepted. Operating under SFE System 100 Pre-Entry
routine, Calling Address Validation Routine module is executing a set of
instructions designed to validate the API function 132 of Fig. 1 calling
address (caller Routine). Caller Routine also includes caller Application
Program Interface, caller system call, caller library call and the like.
Calling Address Validation Routine module commences its
operation by reading the caller Routine return address from the
Procedure Activation Record (stack frame) which is on the user stack
segment (step 191). The stack frame is a dynamic area of the process
stack segment used as a control area for function calls. The process
stack segment is a dynamic area of memory belonging to a process. In
step 192 the caller Routine calling address is calculated (step 192) and
with the help of the data in Process Valid Address Range List it is
examined if the said caller Routine calling address is within valid
address range limits (step 194). In step 196 it is determined whether
the calling address valid or non-valid. To calculate if said caller Routine
calling address is within said valid address range limit said caller Routine calling address is matched with said valid address range limit.
If said caller Routine calling address is within said valid address range
than caller Routine calling address is valid. Next, Calling Address
Validation Routine module by Pre-Entry routine or the like notifies SFE
Server 116 or the like about the test result (step 198 and step 200).
It will be appreciated to by persons skilled in the art that in
this illustrated embodiment of the present invention any unauthorized or
illegal system call or library call originating from memory areas out of
active process address space memory device 118, 120, 122 and the
like of Fig. 1 will be detected and optionally their execution will be
prevented by SFE System 100.
Reference is now made to Fig. 7 which is a high-level flow
diagram of the Calling Address Validation Routine module relating to an
another embodiment of the present invention.
In the embodiment thereof Calling Address Validation Routine
module commences its operation by reading the caller return address
from the Procedure Activation Record (stack frame) on the process
stack segment (step 202). It will be appreciated that reading caller
Routine return address is significantly faster and more accurate. In step
204 the caller Routine calling address is calculated and it is examined /37095
with the help of system-level structures to determine whether the calling
address is inside the address limits of the process stack segment (step
206). Such determination is accomplished by comparing said caller
Routine calling address with address limits of said process stack
segment. Next, Calling Address Validation Routine module by
Pre-Entry routine or the like notifies SFE Server 116 or the like about
the result of the examination (step 210 and step 212).
It will be appreciated by persons skilled in the art that in this
further embodiment of the invention any unauthorized or illegal system
call or library call originating from the process stack segment structure
will be detected and optionally prevented by SFE System 100.
Additional advantages will readily occur to the person skilled
in the art. The invention, in its broader aspects is, therefore, not limited
to the specific details, representative methods, systems and examples
shown and described. It will be furφer appreciated by persons skilled
in the art that the present invention is not limited to what has been
particularly shown and described hereinabove. Rather the scope of the
applicant's general inventive concept and the claims which follow.

Claims

/37095 CLAIMS
1. In a computer system running an operating system platform, said
operating system including a kernel space and a process space,
said process space including a user application running in process
space, said user application operative to intercept system calls, a
method of secure function execution, said method comprising the
step of:
examine said intercepted system call validity by comparing said
intercepted system call originating address with range of
process valid addresses associated with said process from
which said intercepted system call originated.
2. The method of claim 1 , further comprising the step of:
providing notification as to the validity of said intercepted system
call.
3. The method of claim 1 , further comprising the step of:
terminating said intercepted system call.
4. The method of claim 2, further comprising the step of: /37095
terminating said intercepted system call.
5. The method of claim 1 , further comprising the steps of:
responsive to process creation inserting application program
interface interception module into said created process;
responsive to process creation updating process valid addresses
table.
6. The method of claim 1 , further comprising the step of:
responsive to process termination updating process valid
addresses table;
7. In a computer system running an operating system platform, said
operating system including a kernel space and a process space,
said process space including a user application running in process
space, said user application operative to intercept library calls, a
method of secure function execution, said method comprising the
step of:
examine said intercepted library call validity by comparing said
intercepted library call originating address with range of
process valid addresses associated with said process from 37095
which said intercepted library call originated.
8. The method of claim 7, further comprising the step of:
providing notification as to the validity of said intercepted library
call.
9. The method of claim 7, further comprising the step of:
terminating said intercepted library call.
10. The method of claim 8, further comprising the step of:
terminating said intercepted library call.
11. The method of claim 7, further comprising the steps of:
responsive to system call loading dynamic link library hooking and
patching library routines associated with said dynamic link
library;
responsive to system call unloading dynamic link library updating
process valid addresses table;
12. In a computer system running an operating system platform, said
operating system including a kernel space and a process space, 7095
said process space including a user application running in process
space, said user application operative to system and function calls,
said system or function call intercepted, a method of secure function
execution, said method comprising the steps of:
receiving caller routine return address from said process memory
device;
determining whether caller routine address is valid by comparing
said caller address routine with process valid address table.
13. The method of claim 12, further comprising the step of:
providing notification as to the validity of said caller routine return
address.
14. The method of claim 12, further comprising the step of:
performing user predetermined acts associated with said validity of
caller routine address.
15. The method of claim 12, wherein the step of receiving caller
routine return address from said process memory device further
comprises the step of: determining said caller routine calling address by determining the address preceding said caller routine address.
16. In a computer system running an operating system platform, said
operating system including a kernel space and a process space,
said process space including a user application running in process
space, said user application operative to system and function calls,
said system or function call intercepted, a method of secure function
execution, said method comprising the steps of:
receiving caller routine return address from said process memory
device;
determining whether caller routine address is valid by comparing
said caller address routine with associated process stack
address area.
17. The method of claim 16, further comprising the step of:
providing notification as to the validity of said caller routine return
address.
18. The method of claim 16, further comprising the step of: performing user predetermined acts associated with said validity of
caller routine address.
9. The method of claim 16, wherein the step of receiving caller
routine return address from said process memory device further
comprises the step of:
determining said caller routine calling address by determining the
address preceding said caller routine address.
20. The method of secure function execution as substantially
described hereinabove.
21. The method of secure function execution as illustrated in any of
the drawings.
For the Applicant
Soroker - Agmon, Law Offices
PCT/US2000/031032 1999-11-14 2000-11-10 Method and system for intercepting an application program interface WO2001037095A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
AU15986/01A AU1598601A (en) 1999-11-14 2000-11-10 Method and system for intercepting an application program interface
JP2001539121A JP2003515219A (en) 1999-11-14 2000-11-10 Method and system for inhibiting application program interface
EP00978530A EP1236114A1 (en) 1999-11-14 2000-11-10 Method and system for intercepting an application program interface

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
IL132916 1999-11-14
IL13291699A IL132916A (en) 1999-11-14 1999-11-14 Method and system for intercepting an application program interface
US09/561,395 US6823460B1 (en) 1999-11-14 2000-04-28 Method and system for intercepting an application program interface
US09/561,395 2000-04-28
CA002386100A CA2386100A1 (en) 1999-11-14 2002-05-13 Method and system for intercepting application program interface

Publications (1)

Publication Number Publication Date
WO2001037095A1 true WO2001037095A1 (en) 2001-05-25

Family

ID=72714002

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2000/031032 WO2001037095A1 (en) 1999-11-14 2000-11-10 Method and system for intercepting an application program interface

Country Status (5)

Country Link
EP (1) EP1236114A1 (en)
JP (1) JP2003515219A (en)
AU (2) AU1598601A (en)
CA (1) CA2386100A1 (en)
WO (1) WO2001037095A1 (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1361496A2 (en) * 2002-05-06 2003-11-12 Symantec Corporation Alteration of executable code module load locations
WO2004075060A1 (en) * 2003-02-21 2004-09-02 Tabei, Hikaru Computer virus detection device
WO2006047148A1 (en) * 2004-10-25 2006-05-04 Matsushita Electric Industrial Co. Ltd. Security architecture and mechanism to access and use security components in an operating system
CN100346611C (en) * 2005-06-30 2007-10-31 西安交通大学 Invading detection method based on stack pattern in Linux environment
WO2008056944A1 (en) * 2006-11-07 2008-05-15 Softcamp Co., Ltd. Confirmation method of api by the information at call-stack
US7624449B1 (en) 2004-01-22 2009-11-24 Symantec Corporation Countering polymorphic malicious computer code through code optimization
US7739740B1 (en) 2005-09-22 2010-06-15 Symantec Corporation Detecting polymorphic threats
EP2840497A4 (en) * 2012-04-19 2015-11-11 Uni Politècnica De Catalunya Method, system and an executable piece of code for the virtualisation of a hardware resource associated with a computer system

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2005029328A1 (en) * 2003-09-18 2007-11-15 有限会社 電机本舗 Operating system and recording medium recording the same
US8079039B2 (en) * 2007-03-09 2011-12-13 Microsoft Corporation Isolating, managing and communicating with user interface elements
CN102799493A (en) * 2012-06-21 2012-11-28 北京伸得纬科技有限公司 Method for intercepting target progress with self-protection
KR101244731B1 (en) * 2012-09-11 2013-03-18 주식회사 안랩 Apparatus and method for detecting malicious shell code by using debug event
CN103970559B (en) * 2013-02-05 2017-09-29 北京壹人壹本信息科技有限公司 A kind of equipment loading method and device based on android system
JP7036106B2 (en) 2017-02-22 2022-03-15 日本電気株式会社 Information processing equipment, information processing system, monitoring method, and program

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5577209A (en) * 1991-07-11 1996-11-19 Itt Corporation Apparatus and method for providing multi-level security for communication among computers and terminals on a network
US5826014A (en) * 1996-02-06 1998-10-20 Network Engineering Software Firewall system for protecting network elements connected to a public network
US5832228A (en) * 1996-07-30 1998-11-03 Itt Industries, Inc. System and method for providing multi-level security in computer devices utilized with non-secure networks
US6067620A (en) * 1996-07-30 2000-05-23 Holden; James M. Stand alone security device for computer networks
US6088804A (en) * 1998-01-12 2000-07-11 Motorola, Inc. Adaptive system and method for responding to computer network security attacks
US6185689B1 (en) * 1998-06-24 2001-02-06 Richard S. Carson & Assoc., Inc. Method for network self security assessment

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5974549A (en) * 1997-03-27 1999-10-26 Soliton Ltd. Security monitor

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5577209A (en) * 1991-07-11 1996-11-19 Itt Corporation Apparatus and method for providing multi-level security for communication among computers and terminals on a network
US5940591A (en) * 1991-07-11 1999-08-17 Itt Corporation Apparatus and method for providing network security
US5826014A (en) * 1996-02-06 1998-10-20 Network Engineering Software Firewall system for protecting network elements connected to a public network
US6061798A (en) * 1996-02-06 2000-05-09 Network Engineering Software, Inc. Firewall system for protecting network elements connected to a public network
US5832228A (en) * 1996-07-30 1998-11-03 Itt Industries, Inc. System and method for providing multi-level security in computer devices utilized with non-secure networks
US6067620A (en) * 1996-07-30 2000-05-23 Holden; James M. Stand alone security device for computer networks
US6088804A (en) * 1998-01-12 2000-07-11 Motorola, Inc. Adaptive system and method for responding to computer network security attacks
US6185689B1 (en) * 1998-06-24 2001-02-06 Richard S. Carson & Assoc., Inc. Method for network self security assessment

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1361496A2 (en) * 2002-05-06 2003-11-12 Symantec Corporation Alteration of executable code module load locations
EP1361496A3 (en) * 2002-05-06 2004-04-07 Symantec Corporation Alteration of executable code module load locations
US7155741B2 (en) 2002-05-06 2006-12-26 Symantec Corporation Alteration of module load locations
WO2004075060A1 (en) * 2003-02-21 2004-09-02 Tabei, Hikaru Computer virus detection device
JPWO2004075060A1 (en) * 2003-02-21 2006-06-01 田部井 光 Computer virus judgment method
US7624449B1 (en) 2004-01-22 2009-11-24 Symantec Corporation Countering polymorphic malicious computer code through code optimization
WO2006047148A1 (en) * 2004-10-25 2006-05-04 Matsushita Electric Industrial Co. Ltd. Security architecture and mechanism to access and use security components in an operating system
CN100346611C (en) * 2005-06-30 2007-10-31 西安交通大学 Invading detection method based on stack pattern in Linux environment
US7739740B1 (en) 2005-09-22 2010-06-15 Symantec Corporation Detecting polymorphic threats
WO2008056944A1 (en) * 2006-11-07 2008-05-15 Softcamp Co., Ltd. Confirmation method of api by the information at call-stack
EP2840497A4 (en) * 2012-04-19 2015-11-11 Uni Politècnica De Catalunya Method, system and an executable piece of code for the virtualisation of a hardware resource associated with a computer system

Also Published As

Publication number Publication date
AU4888302A (en) 2002-12-05
AU1598601A (en) 2001-05-30
EP1236114A1 (en) 2002-09-04
AU768758B2 (en) 2004-01-08
CA2386100A1 (en) 2003-11-13
JP2003515219A (en) 2003-04-22

Similar Documents

Publication Publication Date Title
US6412071B1 (en) Method for secure function execution by calling address validation
US8661541B2 (en) Detecting user-mode rootkits
US7779062B2 (en) System for preventing keystroke logging software from accessing or identifying keystrokes
CN103886252B (en) Software Code Malicious Selection Evaluation Executed In Trusted Process Address Space
US8307432B1 (en) Generic shellcode detection
US5974549A (en) Security monitor
US7251830B1 (en) Process-based selection of virus detection actions system, method and computer program product
US7934261B1 (en) On-demand cleanup system
US20070050848A1 (en) Preventing malware from accessing operating system services
US7823201B1 (en) Detection of key logging software
AU2006210698B2 (en) Intrusion detection for computer programs
US7665139B1 (en) Method and apparatus to detect and prevent malicious changes to tokens
US20070250927A1 (en) Application protection
US7797702B1 (en) Preventing execution of remotely injected threads
US8539578B1 (en) Systems and methods for defending a shellcode attack
US20070079375A1 (en) Computer Behavioral Management Using Heuristic Analysis
US7251735B2 (en) Buffer overflow protection and prevention
KR20180032566A (en) Systems and methods for tracking malicious behavior across multiple software entities
US9659173B2 (en) Method for detecting a malware
EP1236114A1 (en) Method and system for intercepting an application program interface
EP2038753A1 (en) Identifying malware in a boot environment
CN110119619B (en) System and method for creating anti-virus records
US8910283B1 (en) Firmware-level security agent supporting operating system-level security in computer system
WO2019133637A1 (en) Detection of exploitative program code
EP1236115A1 (en) Method for secure function execution by calling address validation

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AU CA JP

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
ENP Entry into the national phase

Ref country code: JP

Ref document number: 2001 539121

Kind code of ref document: A

Format of ref document f/p: F

WWE Wipo information: entry into national phase

Ref document number: 2000978530

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 2000978530

Country of ref document: EP

WWW Wipo information: withdrawn in national office

Ref document number: 2000978530

Country of ref document: EP