US20070260597A1 - Dynamic search engine results employing user behavior - Google Patents

Dynamic search engine results employing user behavior Download PDF

Info

Publication number
US20070260597A1
US20070260597A1 US11/510,524 US51052406A US2007260597A1 US 20070260597 A1 US20070260597 A1 US 20070260597A1 US 51052406 A US51052406 A US 51052406A US 2007260597 A1 US2007260597 A1 US 2007260597A1
Authority
US
United States
Prior art keywords
user
search
keywords
documents
search result
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11/510,524
Inventor
Mark Cramer
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SURF CANYON
Original Assignee
SURF CANYON
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SURF CANYON filed Critical SURF CANYON
Priority to US11/510,524 priority Critical patent/US20070260597A1/en
Assigned to SURF CANYON reassignment SURF CANYON ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CRAMER, MARK
Priority to PCT/US2007/067799 priority patent/WO2007130914A2/en
Publication of US20070260597A1 publication Critical patent/US20070260597A1/en
Priority to US12/652,004 priority patent/US8095582B2/en
Priority to US13/315,199 priority patent/US20120078710A1/en
Priority to US14/018,791 priority patent/US8924378B2/en
Priority to US14/552,320 priority patent/US9418122B2/en
Priority to US15/053,834 priority patent/US20160350428A1/en
Priority to US15/207,277 priority patent/US20170039285A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241Advertisements
    • G06Q30/0242Determining effectiveness of advertisements
    • G06Q30/0244Optimization
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/953Querying, e.g. by the use of web search engines
    • G06F16/9535Search customisation based on user profiles and personalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241Advertisements
    • G06Q30/0251Targeted advertisements
    • G06Q30/0255Targeted advertisements based on user history
    • G06Q30/0256User search
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99931Database or file accessing
    • Y10S707/99933Query processing, i.e. searching

Definitions

  • the present invention relates to search engines which generate results with improved relevance by monitoring user actions.
  • Search engines are designed to explore data communications networks for documents of interest to a given user and then generate listings of results based on those documents identified in that search.
  • the user specifies this interest by inputting a query, expressed as a “keyword” or set of “keywords,” into the search engine.
  • the keywords are then compared with terms from documents previously indexed by the search engine in order to produce a set of matched documents.
  • the matched documents are ranked by employing any number of different algorithms designed to determine the order with which documents might be relevant to the user. The objective is to quickly point the user toward those matched documents with the greatest likelihood of producing satisfaction.
  • Search engines have thus begun employing many strategies in an attempt to combat this problem. Primarily these consist of improving the algorithms that rank the relevancy of matched documents so that, as the quantity of results increases, those most relevant will still be easily accessible at the top of the list.
  • Some of the major strategies consist of focusing on specific vertical segments, using artificial intelligence to perform contextualized searches, employing personalization, leveraging psychographic, demographic and geographic information and mining the search behaviors of previous users. (Using the behavior of previous users to predict the relevancies of documents for future users has been covered by a number of U.S. patents and applications: 2006/0064411 A1 entitled “Search engine using user intent,” U.S. Pat. No. 6,738,764 B2 entitled “Apparatus and method for adaptively ranking search results,” and U.S. Pat. No. 6,370,526 B1 entitled “Self-adaptive method and system for providing a user-preferred ranking order of object sets,” to name a few.)
  • search engine capable of discerning the intent of the specific user currently conducting a search, at the moment that search is being conducted, in order to dynamically modify the search results “on the fly” with the purpose of ranking the matched documents in the most relevant order possible for that user.
  • the present invention provides a system for using the behavior of a specific user to dynamically modify search results “on the fly” while the actual search is being conducted.
  • the search engine of the preferred embodiment compiles information with respect to the behavior of the user currently conducting a search in order to infer the intent and interests of that user thereby enabling the search engine to present more pertinent results by dynamically altering, in real time, the relevancies, and thus order, of the matched documents.
  • a software application runs as a user interface between a user and a standard third-party search engine, or multiple third-party search engines with the user selecting the preferred. Since the initial results are pulled from the underlying engine they naturally take advantage of all of the technologies and strategies, such as the examples given above, which went into determining the relevancies and ordering of the matching documents in that initial list.
  • the system infers the intent of the current user by collecting data regarding that user's behavior.
  • the system infers the user's intent from information gathered by virtue of the user clicking on documents during a search.
  • other aspects of user behavior such as subsequent clicks on links within documents, time spent looking at different documents (“dwell time”), time spent looking at domains associated with different documents, downloads, transactions, cursor movements, scrolling and highlighting of text, images or other information, are also monitored and used to infer the intent and interests of the user.
  • the inferred intent of the user is characterized by using subordinate keywords.
  • Subordinate keywords as opposed to traditional primary keywords, are keywords that are identified as important to the search, but are not necessarily essential for a matched document. They are automatically generated by the system from a variety of places, such as documents clicked on by the user during the search process as well as documents that are ignored or skipped by the user.
  • the system will then use the subordinate keywords to dynamically re-rank the matched documents “on the fly” to continuously provide the user with the most relevant results possible.
  • the system will additionally use the subordinate keywords to dynamically alter any sponsored links in order to best reflect the intent and interests of the user and, as such, provide the most relevant advertisements and as a result enhance the revenue-generating capability of the system.
  • the outcome is a dynamic results search engine that is not only capable of taking advantage of all of the intelligence and technology of the underlying search engine that went into generating the initial results, but better equipped to help users find the documents they desire by assisting them in navigating increasingly ponderous lists of matched documents in search results.
  • FIG. 1 is a diagram of one embodiment of a system incorporating the present invention.
  • FIG. 2A is a diagram of one embodiment of the home page.
  • FIG. 2B is a diagram of one embodiment of the initial search results.
  • FIG. 2C is a diagram of one embodiment of the re-ranked search results.
  • FIG. 2D is a diagram of one embodiment of the search results re-ranked again.
  • FIG. 3 is a flowchart of one embodiment of the invention.
  • FIG. 4 is a flowchart of one embodiment of subordinate keyword generation and re-ranking of results.
  • FIG. 5 is a diagram of one embodiment of the advanced search results displaying subordinate keywords.
  • the preferred embodiment of the present invention operates on the internet, and more specifically the World Wide Web.
  • the present invention is not limited to the internet, the World Wide Web or any other particular network architecture, software or hardware which may be described herein.
  • the invention is appropriate for any other network architectures, hardware and software.
  • the following description relates to an embodiment utilizing the internet and related protocols, other networks and protocols, for example, for use with interactive TVs, cell phones, personal digital assistants and the like, can be used as well.
  • Website refers to a collection of content. Website content is often transmitted to users via one or more servers that implement basic internet standards. “Website” is not intended to imply a single geographic or physical location but also includes multiple geographically distributed servers that are interconnected via one or more communications systems.
  • the term “document” is defined broadly and includes any type of content, data or information contained in computer files and websites. Content stored by servers and/or transmitted via the communications networks and systems described herein may be stored as a single document, a collection of documents or even a portion of a document. The term “document” is not limited to computer files containing text but also includes files containing graphics, audio, video and other multimedia data. Documents and/or portions of documents may be stored on one or more servers.
  • click or “click-through” is defined broadly and refers to clicking on a hyperlink included within search result listings to view an underlying document or website.
  • clicking on a link or button, or pressing a key to provide a command or make a selection, may also refer to using other input techniques such as voice input, pen input, mousing or hovering over an input area or the like.
  • the dynamic results search engine of the preferred embodiment compiles information with respect to the behavior of the user currently conducting a search in order to infer the interests and intent of that user thereby enabling the search engine to present more pertinent results by dynamically altering the relevancies, and thus order, of the matched documents “on the fly.”
  • the categories of user behavior acquired may include search terms that resulted in click-throughs to particular webpages, websites and sub-domains visited, dwell time, and actions taken at the webpages including document downloads and financial transactions.
  • FIG. 1 is a high-level illustration of a preferred embodiment of a system incorporating the present invention.
  • Documents on servers or computers 110 - 118 on a data communication network 140 are scanned and indexed by search engines on computers and associated databases 120 - 128 .
  • search engines on computers and associated databases 120 - 128 .
  • Google, Yahoo!, MSN and Ask are examples of popular internet search engines, but most any web site that enables users to search documents on a data communications network and retrieve relevant results can be considered a search engine.
  • An embodiment of the present invention provides a server 132 and database 130 which plug into the data communications network and then uses the behavior of current users to dynamically alter the results provided by one, if not several, of the other search engines.
  • this system will focus on a website that takes results from other search engines that reside on the internet, however, another embodiment of the system would involve incorporating the present invention directly into one of the other search engines 120 - 128 . Rather than collecting the search results via a data communications network, the system can gather the results directly from the search engine and then operate accordingly. As discussed briefly below, this embodiment would actually offer some advantages in terms of modifying the rankings of the matched documents.
  • Another embodiment of the system would involve software, which could be made available for download, that resides on the users' computers or terminals 100 - 108 . Now, rather than going to the website of the invention, users can go directly to their search engine of choice and the process of dynamically altering search results will be performed by the software located on the users' computers. Yet another embodiment of the invention involves utilizing its own search engine, as opposed to that of a third-party, should one be available.
  • FIG. 2A is a diagram of a home page of a preferred embodiment of the invention. While other information and links can be present, the principal purpose of the home page is to enable the user to enter a keyword or set of keywords 202 representing the user's query before clicking on a button 204 to request that the search engine retrieve the results.
  • FIG. 2B is a diagram of an example of a search results page depicting what the initial search results for the keyword “Washington” might look like.
  • the invention pulls the results 220 - 227 from another search engine, using publicly available APIs, or some other method, such as, for example, screen scraping, before presenting them to the user.
  • the set of matched documents, and the order in which they are presented, is identical to that of the underlying search engine. It is in this way that the present invention is able to benefit from all of the technologies and strategies used by the underlying search engine when producing the initial search results.
  • Sponsored links 230 - 238 are also made available for the purposes of generating revenue for the system and enabling advertisers to offer their products and services.
  • a third-party ad delivery system such as AdSense from Google, would be one way to accomplish delivering targeted sponsored links.
  • Third-party ad delivery systems either accept keyword submissions or scan the content of a given web page, the search results page in this case, before returning the most relevant ads in their networks. In this way the ads delivered will, to the extent possible, reflect the intent of the current user.
  • Another embodiment would work directly with advertisers by enabling them to purchase keywords before integrating their sponsored links where appropriate.
  • a hybrid approach involving the implementation of a third-party ad delivery system along with working directly with some advertisers, would be yet another embodiment.
  • buttons such as a next results button 216 , are available to enable users to navigate to subsequent pages of results or back to previous pages of results. It should also be noted that there is nothing preventing an embodiment of the system from offering supplemental information on the search results page, as is often the case with search engines, such as related popular queries, suggested spellings or links to maps and stock quotes.
  • FIG. 2C is a diagram of an example of how the first modified results might look after the user clicks on the document “George Washington University” 226 from the initial results as depicted in FIG. 2B and then returns to the search results page.
  • the initial set of matched documents remains the same yet their order has changed based on the intelligence gathered as a consequence of the user's click on the document “George Washington University” 226 .
  • the system has determined that the user is most likely interested in Universities named “Washington” as opposed to George Washington the man or Washington the city or state.
  • the “Access Washington Home Page” document 220 which was initially at the top of the list, has fallen to the sixth position 255 because the system has deemed, based on the user's behavior, that the relevancy of this document has declined.
  • the “University of Washington” document 221 which was initially in the second position, has now moved to the top 250 because the system has deemed, again based on the behavior of the user, that the relevance of this document has increased. This process continues, for all practical purposes, for each document returned in the initial search results. (A discussion regarding practical computation limits is below.)
  • the sponsored links 260 - 268 and their positions also change to more accurately match the intent of the user as deduced by the system based on the user's behavior. While the sponsored link “Visitor Guide Washington” 238 was present in the fifth position on the page with the initial search results in FIG. 2B , it has been removed. Furthermore, the sponsored link “Online University Courses” 264 , while not even present in the initial results, is now in the third position. In the preferred embodiment, the sponsored links provided and the order in which they are displayed will be adjusted continuously, once again, based on the user's behavior. Not only will this provide a benefit to the user as more relevant sponsored links are displayed, but the revenue-generating ability of the system will be enhanced as users will be more inclined to click on the more relevant advertising.
  • FIG. 2D is a diagram of an example of how the second modified results might look after the user clicks on the document “George Washington University—Official Athletic Site” 256 in FIG. 2C and then returns again to the search results page.
  • the order of the search results have once again changed based on this new intelligence gathered with respect to the behavior of the user.
  • the system has determined that the user is most likely interested in athletics at George Washington University.
  • the “University of Washington” document 287 has now moved to the eighth position after initially being in position two 221 in FIG. 2B and then being moved, subsequent to the user's first action, to position one 250 in FIG. 2C . While the system initially increased the relevance of this document after the first action, it was then decreased after the user's second action.
  • the document “George Washington University” 280 has now moved to the top spot after initially being placed in the seventh position 226 in FIG. 2B and then promoted, subsequent to the user's first action, to the second position 251 in FIG. 2C . This process continues, for all practical purposes, for each document returned in the first modified search results in FIG. 2C .
  • the sponsored links 290 - 298 have, once again, changed based on the behavior of the user. “Try eBay” 294 and “‘Tuff Tear’ Paper Numbers” 296 did not appear with the initial search results FIG. 2B or the search results presented after the user's first action FIG. 2C .
  • FIG. 3 is a flowchart of one embodiment of the invention.
  • the user's experience begins on the invention's home page 300 as depicted in FIG. 2A .
  • the system After entering a query, expressed as a keyword or set of keywords, the system produces an initial set of search results 302 as depicted in FIG. 2B .
  • These results can be supplied by any third-party search engine or internally should the system comprise its own search engine.
  • the system begins collecting information regarding the user's behavior in an effort to divine the user's intent and interests and then adjust the results accordingly.
  • Data regarding user behavior immediately following the initial search comes from anything related to the activity of the user, including, but not limited to, clicks on various links in the search results as well as subsequent clicks on links within documents, skipped links in the search results, dwell times, time spent looking at documents from specific domains, resources accessed, transactions conducted, purchases made, orders placed, sessions created, documents downloaded, cursors moved, pages scrolled or text, images or other information highlighted, or any combination thereof.
  • clicks on various links in the search results as well as subsequent clicks on links within documents, skipped links in the search results, dwell times, time spent looking at documents from specific domains, resources accessed, transactions conducted, purchases made, orders placed, sessions created, documents downloaded, cursors moved, pages scrolled or text, images or other information highlighted, or any combination thereof.
  • the process continues by the user taking some form of action 304 such as selecting a document or going to the next page of results.
  • action 304 such as selecting a document or going to the next page of results.
  • the user is taken to that document so that it can be reviewed.
  • the system will simultaneously take information collected regarding the user's behavior to re-rank the initial search results 308 . If the user finds the desired document 310 on the first try, then the user is satisfied and the process is completed. However, should the user return to the search results page to continue the search process, the new search results, having been re-ranked while the user was away, will be displayed 312 as depicted in FIG. 2C .
  • the process then proceeds with the user performing different actions while the system continues to collect data regarding the user's behavior in order to deduce the user's intent and then appropriately re-rank the search results. Each time the user returns to the search results page, the matched documents will have been re-ranked based on new intelligence gleaned from the user's behavior.
  • the primary mechanism for expressing the deduced intent of the user is through the use of “subordinate” keywords.
  • Users typically execute queries with search engines by submitting a set of “primary” keywords.
  • These primary keywords are matched by the search engines with their sets of indexed documents to produce lists of results which are then prioritized using any number of different relevancy algorithms.
  • the matched documents produced must, in one way or another, contain all of the primary keywords submitted for the query. In other words, the primary keywords are “all or nothing;” those documents that do not include the complete set of primary keywords are excluded.
  • Subordinate keywords are keywords that are identified as important but are not necessarily essential to the query. They enable the system to give preferential treatment to (i.e. increase the relevancy of) documents that contain a subset of those keywords without necessarily eliminating those that do not.
  • search engines offer “advanced” search functionality that enables users to specify, for example, keywords that are not to appear or a set of keywords where at least one must be present, these techniques are complex and, even with a bit of manipulation, cannot be used to emulate the functionality or utility of subordinate keywords. Users who are sophisticated enough to avail themselves of the advanced search functionality commonly offered by search engines will still receive significant advantages from the present invention.
  • FIG. 4 is a flowchart of one embodiment of the process of generating subordinate keywords and re-ranking search results.
  • the system when the user takes some sort of action 400 such as selecting a document or continuing to another page of results, the system generates subordinate keywords which are then added to the list of any that might already exist (step 402 ). Based on where the subordinate keywords appear and with what frequency, the system then assigns weights to, or adjusts the weights of, each subordinate keyword (step 404 ). These subordinate keywords, with their associated weights, are then used to adjust the relevancies of the documents in the search results (step 406 ).
  • the new relevancies will be a function of the old relevancies in addition to a function of the weights, frequencies and locations of appearances of the subordinate keywords in each of the matched documents.
  • the new relevancies are then used to dynamically re-rank the documents in the search results (step 408 ) with the effect of moving the most relevant documents up while pushing the least relevant down.
  • Subordinate keywords are automatically generated by the system from a variety of places, including, but not limited to, links clicked on by the user, other links associated with the document such as links pointing to the document, “descriptive text” associated with each document in the search results, meta-tags connected to viewed documents, and prominent words and phrases in viewed documents. (As is common practice with search engines, “stop words,” defined as those words which are so common that they are useless to index, are ignored.) A thesaurus can also be used to generate similar words and phrases that might be of interest to the user. Since subordinate keywords are simply an expression of important ideas, and do not as such eliminate any matched documents from the results of a query, they can be employed with abundance. In fact, the more subordinate keywords are generated from the user's behavior, the more likely the system is to find the most relevant documents and move them to the top of the search results.
  • one embodiment of the system might generate subordinate keywords from every word in the title and display text of any document selected by virtue of a user's click.
  • the title and display text for document 226 are, respectively, “George Washington University” and “The George Washington University is located four blocks from the White House and was created by an Act of Congress in 1821.” Therefore, if N represents the number of subordinate keywords generated, sk n represents the n th subordinate keyword and s n represents the number of times that the n th subordinate keyword appears in the title and display text of the selected document, then TABLE I is a demonstration of the subordinate keywords generated from clicking on document 226 :
  • the next step is to assign “weights” to each subordinate keyword.
  • Some subordinate keywords will undoubtedly be more important than others and assigning weights will enable the system to more accurately express the deduced intent of the user.
  • the weight of each subordinate keyword is determined based on a number of factors, including, but not limited to, the placements of the keyword and frequencies with which it appears in the links, descriptive text, meta-tags or any other information associated with documents referred to by the user, including the documents themselves.
  • subordinate keywords can even be deemed to have negative weights, meaning that they represent ideas in which the user is not interested.
  • negative keywords are generated from the titles and display texts of documents that are passed over by the user. For example, if the user were to click on the fourth document in a list of results, it can be inferred from the user's behavior that there is little or no interest in the information presented in the first three results. As such, any subordinate keywords present in the titles and display texts of the first three documents can be given a negative weight.
  • subordinate keyword weights are further adjusted by soliciting feedback from the user with respect to each document viewed.
  • the system can adjust the extent to which the weights of the subordinate keywords associated with that document are raised or lowered.
  • other information regarding the user's behavior such as dwell time and any sort of activity, can be used to infer to what extent viewed documents are interesting to the user.
  • the weight of a given subordinate keyword will be a function that takes into account the locations and frequencies of its appearances. Subordinate keywords that appear in some places, such as titles or display texts, may be given more weight than if they had appeared elsewhere, such as buried in selected documents. Furthermore, subordinate keywords that appear in important documents may be given more weight than if they had appeared in less important documents.
  • One embodiment of the system might make the weight of a subordinate keyword a function of the number of times it appears in the titles or display texts of documents that have been selected by virtue of a user click.
  • the following formula, using the arctangent function to provide a mechanism for having the weights asymptotically approach a given value as the frequency of appearances increases, could be used to generate weights in the range of ⁇ 100 to +100:
  • TABLE II indicates the weights that would be associated with each subordinate keyword by employing the formula above:
  • the data is utilized to re-rank the matched documents in the search results.
  • Documents are increased (or decreased) in importance and moved up (or down) in the prioritization of the search results based on their association with the subordinate keywords.
  • a ranking algorithm must be developed and then tuned to specify the impact that each subordinate keyword has on the movement of the documents in the search results.
  • the ranking function will run through the search results and adjust the rank of each matched document based on which subordinate keywords are associated with each document, taking into account the weights of each as well as where exactly they are found.
  • the ranking function will, naturally, also take into account the previous rank of the document, helping to reflect, in some way, the intelligence that went into producing the initial order of the search results.
  • the weights of the subordinate keywords were based on where and how often those keywords appeared in relation to the selected, or skipped, documents, the movement of matched documents in the search results will similarly depend on where the subordinate keywords appear.
  • a document with a large number of high-weight subordinate keywords in its title, display text and meta-tags will perhaps move much more dramatically than a document with a few low-weight subordinate keywords buried deep within the text of the document. It should also be noted that the presence of subordinate keywords with positive weights will increase the relevancy of the document, moving it up in the rankings, while the presence of subordinate keywords with negative weights will decrease the relevancy and have the opposite effect.
  • the ranks of the matched documents serve as a proxy for relevancy.
  • the underlying search engine were to share the calculated relevancy scores of the matched documents, via some method of communication such as an API, or if the invention was actually incorporated into the underlying search engine itself, then those relevancy scores could be used for r m , in place of the rank, potentially increasing the effectiveness of the system.
  • TABLE IV is a depiction of the first eight search results as demonstrated in FIG. 2B :
  • One embodiment of the system might use a ranking function that makes the new rank of a document equal to its previous rank plus some function of the subordinate keywords that appear in the document's title and display text.
  • W r and W s represent constant weights and Er and Es represent constant exponents
  • a ranking function using the sgn( ) and absolute value functions to handle negative subordinate keyword weights, could be displayed as follows:
  • the values of the constants in the ranking function will be developed by careful analysis of empirical user data. The objective is to determine these values in order to optimize the movement of documents and minimize the amount of searching required by the user to find the desired information.
  • One embodiment of the invention uses empirical user data as it is collected to refine the values of the constants in real time. By identifying the end of a successful search, possibly but not necessarily with the help of feedback from the user, the system could, over time, adjust the values of the constants in order to maximize the percentage of searches that end successfully while minimizing the time required to complete a successful search.
  • the constants are actually customized for each user, representing how different users behave differently, and stored in a user profile or cookie.
  • the constants also depend on other information such as the number of matched documents, which underlying search engine is being used, the language of the results, the country where the user is located, or virtually any other variable.
  • the only subordinate keyword from TABLE II found in either the title, t 1 , or display text, d 1 is “locate.”
  • the only subordinate keyword from TABLE II found in either the title, t 2 , or display text, d 2 is “university,” which has a weight of 37.
  • the nine words “located,” “four,” “blocks,” “white,” “house,” “created,” “act,” “congress,” and “1821” all appear once in the display text, d 7 , and have weights of 20.
  • the server hosting the invention software might prohibit running the ranking algorithm on all of the matched documents generated by a query, especially if there are millions of them. Not only is processor speed required to execute all of the calculations, but the server memory might need to be large enough to hold all of the results.
  • the ranking algorithm can be run on a fairly large number of matched documents, the first several hundred or thousand for example, without significantly impacting the effectiveness of the system. Should a determined user page through a large proportion of those re-ranked documents, the system can simply grab the next batch of several hundred or thousand and quickly re-rank those with the previous batch. In any event, it is important to run as many computations as possible in the background while the user is reading or reviewing documents in order to avoid imposing delays on the user.
  • the ranking function can take into account the presence of subordinate keywords in the actual documents, this could additionally require a significant amount of bandwidth and processing power as each document is downloaded and reviewed.
  • a computational and time-saving technique would be to use the power of the underlying search engine, or even another search engine, to accelerate the speed with which subordinate keywords are identified in matched documents.
  • the system can, in the background, run queries using the subordinate keywords, or just the most important thereof to save on computational time, in order to quickly determine which of the matched documents contain the recently generated subordinate keywords. At this point all that is required is a simple corresponding of the initial matched documents with the results generated by the subordinate keyword queries.
  • the ranking function can then quickly take into account the presence of subordinate keywords in the matched documents themselves and adjust the rankings accordingly.
  • FIG. 5 is a diagram of one embodiment of an “advanced search” page of the invention that actually displays the subordinate keywords to the user. While the matched documents 510 - 516 and the sponsored links 520 - 528 are the same, an “information bar” on the left-hand side contains the primary keywords 530 and subordinate keywords 531 - 550 currently employed.
  • the subordinate keywords can be shown in color, with blue representing positive subordinate keywords and red representing negative subordinate keywords. While the actual color is not important, the intensity of the color corresponds with the weight: the more intense the color, the more the weight is either positive or negative. The subordinate keywords would thus be listed from dark blue (very positive) to dark red (very negative).
  • Displaying the subordinate keywords can be beneficial to the user for a couple of reasons: not only does this give insight into how the system is operating, but users can then assist the system in locating relevant documents by either manually removing or promoting specific subordinate keywords. Should the user know that a particular subordinate keyword is not relevant to the query, that keyword can be selected and then removed by pressing the “remove” button 560 . Should the user see a subordinate keyword that is deemed more than just important, but essential, to the query, it can be selected and then promoted to a primary keyword by pressing the “promote” button 562 . Once any set of keywords is promoted to primary the initial search will have to be rerun by returning to the underlying search engine for a new set of results. That being said, all of the remaining subordinate keywords and weights can be carried forward with the new set of search results being adjusted accordingly.
  • “bread crumbs” linking to previous search results pages 570 - 572 will enable the user, if so desired, to go back to previous rankings of matched documents. Additionally, since this system is a novel approach to assisting the user to find relevant documents, the movement of the matched documents might initially be confusing. Nevertheless, a few things can thus be done with the user interface to help ease the transition. “Movement indicators” can be placed immediately to the left of document titles 510 - 517 to indicate how the documents have moved since the last visit to the search results page (+6, +9, +13, etc.).
  • boxes of one color such as blue
  • boxes of another color such as red
  • boxes of another color such as red
  • the sponsored links 520 - 528 also change dynamically based on the deduced intent of the user.
  • a third-party sponsored link service that either accepts keyword submissions or scans the content of a page, can take subordinate keywords into account to deliver relevant advertisements. Whether a third-party sponsored link service is used or not, subordinate keywords, since they are a representation of the deduced intent of the user, should be used to dynamically alter the sponsored links that are displayed.
  • the system should devise techniques for having the selection of sponsored links take the subordinate keywords into consideration. Increasing the accuracy of targeted advertisements that are displayed will have the dual benefit of improving the user experience while increasing the revenue generated by the system.
  • the outcome is thus a dynamic results search engine that continuously changes, updates and reorganizes search results based upon the intent of the user as deduced from the ongoing behavior of the user during the search process.
  • the system takes this information, deduces the intent of the user, generates subordinate keywords and assigns weights to them before then reprioritizing the search results “on the fly.”
  • the end result is a dynamic results search engine better able to assist users in finding desired documents and information.
  • user behavior monitoring is done by a first software module on server 132 in FIG. 1 .
  • the behavior information is provided to a second, re-ranking software module on server 132 .
  • the re-ranking module modifies the rankings of search results received from a third-party search engine and stores the re-ranked results on database 130 .
  • the re-ranking software module also performs the highlighting, showing of ranking movement, displaying of subordinate keywords, and other functions not performed by the monitoring software module or the third-party search engine.
  • the re-ranked search results are supplied to a client browser on the user's computer.
  • the client browser software may perform some of the customized display functions of the invention.
  • the re-ranking software module works in parallel with user actions, performing re-ranking while the user behavior is being monitored.
  • a series of re-ranked results can be created and stored in database 130 as the user clicks through documents. If the user decides that a particular document is not relevant, then the re-ranking module takes the subordinate keywords associated with that document, makes their weights negative, and pushes the document (and others that are similar) down.
  • the user browser may be installed on other devices than a computer, such as a personal digital assistant (PDA), a mobile phone, or any other device.
  • PDA personal digital assistant
  • the display can be modified to fit a smaller form factor, such as by providing the sponsor links before or after a group of search results.
  • audio indications could be used.
  • voice input can be used to remove or promote subordinate keywords, or for any other user input.
  • the present invention could be embodied in other specific forms without departing from the essential characteristics thereof.
  • data based on prior users' experiences and click streams could be used to re-rank the results in real time. This could provide two levels of ranking, (1) a first re-ranking using subordinate keywords as described above, and (2) a re-ranking of the re-ranked results using the webpages found most desirable by previous users doing similar searches.
  • the present invention can use a separate third party search engine, or could be integrated with a search engine.
  • the search engine could be a general search engine that searches the internet, a specialized search engine that searches a particular web site, a database search engine, a meta-search engine that combines the results of multiple other search engines, or any other type of search engine. Accordingly the foregoing description is intended to be illustrative, but not limiting, of the scope of the invention which is set forth in the following claims.

Abstract

A method and apparatus for dynamically modifying search results “on the fly” based on the behavior of the user currently conducting a search. In one embodiment, data regarding user behavior is gathered from virtually any activity, including clicks on links, dwell times, downloads, transactions and cursor movements. Subordinate keywords are generated to reflect the intent of the user as inferred from the user's behavior. Subordinate keywords, as opposed to traditional primary keywords, are keywords that are identified as important to the search, but are not necessarily essential for a matched document. They are automatically generated by the system from a variety of places, such as documents clicked on by the user as well as documents that are skipped. The system uses the subordinate keywords to dynamically re-rank matched documents and advertisements to best reflect the inferred intent of the user in order to continuously provide the most relevant results.

Description

    CROSS-REFERENCES TO RELATED APPLICATIONS
  • This application claims the benefit of U.S. Provisional Patent Application No. 60/796,624, filed May 2, 2006, entitled “Dynamic Search Engine Results Using User Behavior”, which is hereby incorporated by reference in its entirety.
  • BACKGROUND OF THE INVENTION
  • The present invention relates to search engines which generate results with improved relevance by monitoring user actions.
  • Search engines are designed to explore data communications networks for documents of interest to a given user and then generate listings of results based on those documents identified in that search. The user specifies this interest by inputting a query, expressed as a “keyword” or set of “keywords,” into the search engine. The keywords are then compared with terms from documents previously indexed by the search engine in order to produce a set of matched documents. Finally, before being presented, the matched documents are ranked by employing any number of different algorithms designed to determine the order with which documents might be relevant to the user. The objective is to quickly point the user toward those matched documents with the greatest likelihood of producing satisfaction.
  • On the internet (a popular, global data communications network) the number of indexed documents has been growing rapidly due predominantly to improvements in technology and the growth in the quantity of information available. Some queries now return millions of matched documents. As a result, the ability of internet search engines to help users identify documents of particular interest to a given query is being hampered. In other words, while internet users have access to an increasing quantity of potentially relevant information, using keyword queries on search engines to identify those documents that produce satisfaction has become more difficult.
  • Search engines have thus begun employing many strategies in an attempt to combat this problem. Primarily these consist of improving the algorithms that rank the relevancy of matched documents so that, as the quantity of results increases, those most relevant will still be easily accessible at the top of the list. Some of the major strategies consist of focusing on specific vertical segments, using artificial intelligence to perform contextualized searches, employing personalization, leveraging psychographic, demographic and geographic information and mining the search behaviors of previous users. (Using the behavior of previous users to predict the relevancies of documents for future users has been covered by a number of U.S. patents and applications: 2006/0064411 A1 entitled “Search engine using user intent,” U.S. Pat. No. 6,738,764 B2 entitled “Apparatus and method for adaptively ranking search results,” and U.S. Pat. No. 6,370,526 B1 entitled “Self-adaptive method and system for providing a user-preferred ranking order of object sets,” to name a few.)
  • While all of these strategies, used either singularly or in combination, provide some benefit, they are incomplete for the simple reason that they do not take into account the behavior of the specific user conducting the search at the moment the search is actually being conducted. Consequently, there is a lot of room to adjust and improve the relevancies of matched documents by examining the behavior of the current user and then responding accordingly in real time.
  • To illustrate, consider a user who submits a query using the keyword “Washington.” Different algorithms, using the strategies mentioned above or potentially others, can be deployed to determine which documents might have the highest probability of being relevant to that specific user for that particular query. By using only information available prior to the submission, however, there is no way of knowing a priori, with any significant degree of certainty, if the user is more interested in “Washington University,” “George Washington,” or, for that matter, “George Washington University.” Thus, while the search engine, employing whatever algorithms and strategies it deems best, attempts to present the results in the most relevant order possible, without additional information from the user there is a substantial chance that the results will be littered with irrelevant documents.
  • At this point the users' options are limited. They can scan page by page through potentially millions of extraneous matched documents in an attempt to pick out the ones that matter or they can try to identify additional or more specific keywords in an attempt to produce new, and hopefully more relevant, sets of results. Depending on the nature of the search and the ingenuity of the user, this task can often be painstaking and frustrating, if not impossible.
  • There is therefore a need for a search engine capable of discerning the intent of the specific user currently conducting a search, at the moment that search is being conducted, in order to dynamically modify the search results “on the fly” with the purpose of ranking the matched documents in the most relevant order possible for that user.
  • SUMMARY OF THE INVENTION
  • The present invention provides a system for using the behavior of a specific user to dynamically modify search results “on the fly” while the actual search is being conducted. The search engine of the preferred embodiment compiles information with respect to the behavior of the user currently conducting a search in order to infer the intent and interests of that user thereby enabling the search engine to present more pertinent results by dynamically altering, in real time, the relevancies, and thus order, of the matched documents.
  • In one embodiment, a software application runs as a user interface between a user and a standard third-party search engine, or multiple third-party search engines with the user selecting the preferred. Since the initial results are pulled from the underlying engine they naturally take advantage of all of the technologies and strategies, such as the examples given above, which went into determining the relevancies and ordering of the matching documents in that initial list.
  • From there, the system infers the intent of the current user by collecting data regarding that user's behavior. In one embodiment, the system infers the user's intent from information gathered by virtue of the user clicking on documents during a search. In another embodiment, other aspects of user behavior, such as subsequent clicks on links within documents, time spent looking at different documents (“dwell time”), time spent looking at domains associated with different documents, downloads, transactions, cursor movements, scrolling and highlighting of text, images or other information, are also monitored and used to infer the intent and interests of the user.
  • In one embodiment, the inferred intent of the user is characterized by using subordinate keywords. Subordinate keywords, as opposed to traditional primary keywords, are keywords that are identified as important to the search, but are not necessarily essential for a matched document. They are automatically generated by the system from a variety of places, such as documents clicked on by the user during the search process as well as documents that are ignored or skipped by the user.
  • In one embodiment, the system will then use the subordinate keywords to dynamically re-rank the matched documents “on the fly” to continuously provide the user with the most relevant results possible. The system will additionally use the subordinate keywords to dynamically alter any sponsored links in order to best reflect the intent and interests of the user and, as such, provide the most relevant advertisements and as a result enhance the revenue-generating capability of the system.
  • The outcome is a dynamic results search engine that is not only capable of taking advantage of all of the intelligence and technology of the underlying search engine that went into generating the initial results, but better equipped to help users find the documents they desire by assisting them in navigating increasingly ponderous lists of matched documents in search results.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a diagram of one embodiment of a system incorporating the present invention.
  • FIG. 2A is a diagram of one embodiment of the home page.
  • FIG. 2B is a diagram of one embodiment of the initial search results.
  • FIG. 2C is a diagram of one embodiment of the re-ranked search results.
  • FIG. 2D is a diagram of one embodiment of the search results re-ranked again.
  • FIG. 3 is a flowchart of one embodiment of the invention.
  • FIG. 4 is a flowchart of one embodiment of subordinate keyword generation and re-ranking of results.
  • FIG. 5 is a diagram of one embodiment of the advanced search results displaying subordinate keywords.
  • DETAILED DESCRIPTION OF THE INVENTION
  • The preferred embodiment of the present invention operates on the internet, and more specifically the World Wide Web. The present invention, however, is not limited to the internet, the World Wide Web or any other particular network architecture, software or hardware which may be described herein. The invention is appropriate for any other network architectures, hardware and software. Furthermore, while the following description relates to an embodiment utilizing the internet and related protocols, other networks and protocols, for example, for use with interactive TVs, cell phones, personal digital assistants and the like, can be used as well.
  • The functions described herein are performed by programs including executable code or instructions running on one or more general-purpose computers. The functions described herein, however, can also be implemented using special purpose computers, state machines and/or hardwired electronic circuits. The example processes described herein do not necessarily have to be performed in the described sequence and not all states have to be reached or performed.
  • As used herein, the term “website” refers to a collection of content. Website content is often transmitted to users via one or more servers that implement basic internet standards. “Website” is not intended to imply a single geographic or physical location but also includes multiple geographically distributed servers that are interconnected via one or more communications systems.
  • As used herein, the term “document” is defined broadly and includes any type of content, data or information contained in computer files and websites. Content stored by servers and/or transmitted via the communications networks and systems described herein may be stored as a single document, a collection of documents or even a portion of a document. The term “document” is not limited to computer files containing text but also includes files containing graphics, audio, video and other multimedia data. Documents and/or portions of documents may be stored on one or more servers.
  • As used herein, the term “click” or “click-through” is defined broadly and refers to clicking on a hyperlink included within search result listings to view an underlying document or website. The term “clicking on” a link or button, or pressing a key to provide a command or make a selection, may also refer to using other input techniques such as voice input, pen input, mousing or hovering over an input area or the like.
  • The dynamic results search engine of the preferred embodiment compiles information with respect to the behavior of the user currently conducting a search in order to infer the interests and intent of that user thereby enabling the search engine to present more pertinent results by dynamically altering the relevancies, and thus order, of the matched documents “on the fly.” The categories of user behavior acquired may include search terms that resulted in click-throughs to particular webpages, websites and sub-domains visited, dwell time, and actions taken at the webpages including document downloads and financial transactions.
  • FIG. 1 is a high-level illustration of a preferred embodiment of a system incorporating the present invention. Documents on servers or computers 110-118 on a data communication network 140, such as the internet, are scanned and indexed by search engines on computers and associated databases 120-128. On the internet there are currently billions of documents available, with more being added daily. Google, Yahoo!, MSN and Ask are examples of popular internet search engines, but most any web site that enables users to search documents on a data communications network and retrieve relevant results can be considered a search engine. Users on computers or terminals 100-108 seeking information can connect with the search engine of their choice, enter keywords related to their searches, and then retrieve lists of matched documents, normally sorted and ranked by some sort of algorithm that places the ones deemed most relevant at the top. An embodiment of the present invention provides a server 132 and database 130 which plug into the data communications network and then uses the behavior of current users to dynamically alter the results provided by one, if not several, of the other search engines.
  • The description of this system will focus on a website that takes results from other search engines that reside on the internet, however, another embodiment of the system would involve incorporating the present invention directly into one of the other search engines 120-128. Rather than collecting the search results via a data communications network, the system can gather the results directly from the search engine and then operate accordingly. As discussed briefly below, this embodiment would actually offer some advantages in terms of modifying the rankings of the matched documents. Another embodiment of the system would involve software, which could be made available for download, that resides on the users' computers or terminals 100-108. Now, rather than going to the website of the invention, users can go directly to their search engine of choice and the process of dynamically altering search results will be performed by the software located on the users' computers. Yet another embodiment of the invention involves utilizing its own search engine, as opposed to that of a third-party, should one be available.
  • FIG. 2A is a diagram of a home page of a preferred embodiment of the invention. While other information and links can be present, the principal purpose of the home page is to enable the user to enter a keyword or set of keywords 202 representing the user's query before clicking on a button 204 to request that the search engine retrieve the results.
  • FIG. 2B is a diagram of an example of a search results page depicting what the initial search results for the keyword “Washington” might look like. Upon receiving a query 212 from a user, expressed as a keyword or set of keywords, the invention pulls the results 220-227 from another search engine, using publicly available APIs, or some other method, such as, for example, screen scraping, before presenting them to the user. The set of matched documents, and the order in which they are presented, is identical to that of the underlying search engine. It is in this way that the present invention is able to benefit from all of the technologies and strategies used by the underlying search engine when producing the initial search results.
  • Sponsored links 230-238 are also made available for the purposes of generating revenue for the system and enabling advertisers to offer their products and services. A third-party ad delivery system, such as AdSense from Google, would be one way to accomplish delivering targeted sponsored links. Third-party ad delivery systems either accept keyword submissions or scan the content of a given web page, the search results page in this case, before returning the most relevant ads in their networks. In this way the ads delivered will, to the extent possible, reflect the intent of the current user. Another embodiment would work directly with advertisers by enabling them to purchase keywords before integrating their sponsored links where appropriate. A hybrid approach, involving the implementation of a third-party ad delivery system along with working directly with some advertisers, would be yet another embodiment.
  • Since the number of matched documents rarely fits on a single page, buttons, such as a next results button 216, are available to enable users to navigate to subsequent pages of results or back to previous pages of results. It should also be noted that there is nothing preventing an embodiment of the system from offering supplemental information on the search results page, as is often the case with search engines, such as related popular queries, suggested spellings or links to maps and stock quotes.
  • FIG. 2C is a diagram of an example of how the first modified results might look after the user clicks on the document “George Washington University” 226 from the initial results as depicted in FIG. 2B and then returns to the search results page. The initial set of matched documents remains the same yet their order has changed based on the intelligence gathered as a consequence of the user's click on the document “George Washington University” 226. The system has determined that the user is most likely interested in Universities named “Washington” as opposed to George Washington the man or Washington the city or state. As a result, the “Access Washington Home Page” document 220, which was initially at the top of the list, has fallen to the sixth position 255 because the system has deemed, based on the user's behavior, that the relevancy of this document has declined. On the other hand, the “University of Washington” document 221, which was initially in the second position, has now moved to the top 250 because the system has deemed, again based on the behavior of the user, that the relevance of this document has increased. This process continues, for all practical purposes, for each document returned in the initial search results. (A discussion regarding practical computation limits is below.)
  • As the relevancies and thus positions of the matched documents change, the sponsored links 260-268 and their positions also change to more accurately match the intent of the user as deduced by the system based on the user's behavior. While the sponsored link “Visitor Guide Washington” 238 was present in the fifth position on the page with the initial search results in FIG. 2B, it has been removed. Furthermore, the sponsored link “Online University Courses” 264, while not even present in the initial results, is now in the third position. In the preferred embodiment, the sponsored links provided and the order in which they are displayed will be adjusted continuously, once again, based on the user's behavior. Not only will this provide a benefit to the user as more relevant sponsored links are displayed, but the revenue-generating ability of the system will be enhanced as users will be more inclined to click on the more relevant advertising.
  • FIG. 2D is a diagram of an example of how the second modified results might look after the user clicks on the document “George Washington University—Official Athletic Site” 256 in FIG. 2C and then returns again to the search results page. The order of the search results have once again changed based on this new intelligence gathered with respect to the behavior of the user. The system has determined that the user is most likely interested in athletics at George Washington University. Thus, the “University of Washington” document 287 has now moved to the eighth position after initially being in position two 221 in FIG. 2B and then being moved, subsequent to the user's first action, to position one 250 in FIG. 2C. While the system initially increased the relevance of this document after the first action, it was then decreased after the user's second action. The document “George Washington University” 280 has now moved to the top spot after initially being placed in the seventh position 226 in FIG. 2B and then promoted, subsequent to the user's first action, to the second position 251 in FIG. 2C. This process continues, for all practical purposes, for each document returned in the first modified search results in FIG. 2C.
  • The sponsored links 290-298 have, once again, changed based on the behavior of the user. “Try eBay” 294 and “‘Tuff Tear’ Paper Numbers” 296 did not appear with the initial search results FIG. 2B or the search results presented after the user's first action FIG. 2C. The sponsored link “The Lodge at Suncadia” 298, while previously in the second position has been moved down to the fifth position.
  • FIG. 3 is a flowchart of one embodiment of the invention. The user's experience begins on the invention's home page 300 as depicted in FIG. 2A. After entering a query, expressed as a keyword or set of keywords, the system produces an initial set of search results 302 as depicted in FIG. 2B. These results can be supplied by any third-party search engine or internally should the system comprise its own search engine.
  • Once the initial search results are presented, the system begins collecting information regarding the user's behavior in an effort to divine the user's intent and interests and then adjust the results accordingly. Data regarding user behavior immediately following the initial search comes from anything related to the activity of the user, including, but not limited to, clicks on various links in the search results as well as subsequent clicks on links within documents, skipped links in the search results, dwell times, time spent looking at documents from specific domains, resources accessed, transactions conducted, purchases made, orders placed, sessions created, documents downloaded, cursors moved, pages scrolled or text, images or other information highlighted, or any combination thereof. In general, the more time spent looking and conducting activities at a particular website, the more relevant that website is to the user.
  • The process continues by the user taking some form of action 304 such as selecting a document or going to the next page of results. In the case of selecting a document, the user is taken to that document so that it can be reviewed. While the document is being reviewed, the system will simultaneously take information collected regarding the user's behavior to re-rank the initial search results 308. If the user finds the desired document 310 on the first try, then the user is satisfied and the process is completed. However, should the user return to the search results page to continue the search process, the new search results, having been re-ranked while the user was away, will be displayed 312 as depicted in FIG. 2C. The process then proceeds with the user performing different actions while the system continues to collect data regarding the user's behavior in order to deduce the user's intent and then appropriately re-rank the search results. Each time the user returns to the search results page, the matched documents will have been re-ranked based on new intelligence gleaned from the user's behavior.
  • The primary mechanism for expressing the deduced intent of the user is through the use of “subordinate” keywords. Users typically execute queries with search engines by submitting a set of “primary” keywords. These primary keywords are matched by the search engines with their sets of indexed documents to produce lists of results which are then prioritized using any number of different relevancy algorithms. The matched documents produced, however, must, in one way or another, contain all of the primary keywords submitted for the query. In other words, the primary keywords are “all or nothing;” those documents that do not include the complete set of primary keywords are excluded. (Some search engines apply some “fuzziness” to this rule with word stemming and other techniques, and contextualized search engines apply even more “fuzziness” as they attempt to match concepts as opposed to terms, but the basic principal remains.) Subordinate keywords, on the other hand, are keywords that are identified as important but are not necessarily essential to the query. They enable the system to give preferential treatment to (i.e. increase the relevancy of) documents that contain a subset of those keywords without necessarily eliminating those that do not.
  • While many search engines offer “advanced” search functionality that enables users to specify, for example, keywords that are not to appear or a set of keywords where at least one must be present, these techniques are complex and, even with a bit of manipulation, cannot be used to emulate the functionality or utility of subordinate keywords. Users who are sophisticated enough to avail themselves of the advanced search functionality commonly offered by search engines will still receive significant advantages from the present invention.
  • FIG. 4 is a flowchart of one embodiment of the process of generating subordinate keywords and re-ranking search results. As mentioned previously, when the user takes some sort of action 400 such as selecting a document or continuing to another page of results, the system generates subordinate keywords which are then added to the list of any that might already exist (step 402). Based on where the subordinate keywords appear and with what frequency, the system then assigns weights to, or adjusts the weights of, each subordinate keyword (step 404). These subordinate keywords, with their associated weights, are then used to adjust the relevancies of the documents in the search results (step 406). The new relevancies will be a function of the old relevancies in addition to a function of the weights, frequencies and locations of appearances of the subordinate keywords in each of the matched documents. The new relevancies are then used to dynamically re-rank the documents in the search results (step 408) with the effect of moving the most relevant documents up while pushing the least relevant down.
  • Subordinate keywords are automatically generated by the system from a variety of places, including, but not limited to, links clicked on by the user, other links associated with the document such as links pointing to the document, “descriptive text” associated with each document in the search results, meta-tags connected to viewed documents, and prominent words and phrases in viewed documents. (As is common practice with search engines, “stop words,” defined as those words which are so common that they are useless to index, are ignored.) A thesaurus can also be used to generate similar words and phrases that might be of interest to the user. Since subordinate keywords are simply an expression of important ideas, and do not as such eliminate any matched documents from the results of a query, they can be employed with abundance. In fact, the more subordinate keywords are generated from the user's behavior, the more likely the system is to find the most relevant documents and move them to the top of the search results.
  • For example, one embodiment of the system might generate subordinate keywords from every word in the title and display text of any document selected by virtue of a user's click. In FIG. 2B the title and display text for document 226 are, respectively, “George Washington University” and “The George Washington University is located four blocks from the White House and was created by an Act of Congress in 1821.” Therefore, if N represents the number of subordinate keywords generated, skn represents the nth subordinate keyword and sn represents the number of times that the nth subordinate keyword appears in the title and display text of the selected document, then TABLE I is a demonstration of the subordinate keywords generated from clicking on document 226:
  • TABLE I
    n skn sn
    1 george 2
    2 university 2
    3 located 1
    4 four 1
    5 blocks 1
    6 white 1
    7 house 1
    8 created I
    9 act 1
    10 congress 1
    11 1821 1
  • (It should be noted that “Washington” is not a subordinate keyword because it is a primary keyword 212.)
  • The next step is to assign “weights” to each subordinate keyword. Some subordinate keywords will undoubtedly be more important than others and assigning weights will enable the system to more accurately express the deduced intent of the user. The weight of each subordinate keyword is determined based on a number of factors, including, but not limited to, the placements of the keyword and frequencies with which it appears in the links, descriptive text, meta-tags or any other information associated with documents referred to by the user, including the documents themselves.
  • Depending upon the behavior of the user, subordinate keywords can even be deemed to have negative weights, meaning that they represent ideas in which the user is not interested. In one embodiment of the system, negative keywords are generated from the titles and display texts of documents that are passed over by the user. For example, if the user were to click on the fourth document in a list of results, it can be inferred from the user's behavior that there is little or no interest in the information presented in the first three results. As such, any subordinate keywords present in the titles and display texts of the first three documents can be given a negative weight.
  • In one embodiment, subordinate keyword weights are further adjusted by soliciting feedback from the user with respect to each document viewed. By requesting that the user indicate, on a scale, for example, the usefulness of a document just viewed, the system can adjust the extent to which the weights of the subordinate keywords associated with that document are raised or lowered. In the absence of such feedback, other information regarding the user's behavior, such as dwell time and any sort of activity, can be used to infer to what extent viewed documents are interesting to the user.
  • In any event, the weight of a given subordinate keyword will be a function that takes into account the locations and frequencies of its appearances. Subordinate keywords that appear in some places, such as titles or display texts, may be given more weight than if they had appeared elsewhere, such as buried in selected documents. Furthermore, subordinate keywords that appear in important documents may be given more weight than if they had appeared in less important documents.
  • If In represents the weight of the nth subordinate keyword and an through zn, and possibly beyond, represent the frequency with which skn appears in various places in specific documents, such as titles, descriptive texts, links, meta-tags and so forth, then the following represents a generalized formula for calculating subordinate keyword weights:

  • l n =f(a n)+f 2(b n)+f 3(c n)+ . . .
  • One embodiment of the system might make the weight of a subordinate keyword a function of the number of times it appears in the titles or display texts of documents that have been selected by virtue of a user click. As such, the following formula, using the arctangent function to provide a mechanism for having the weights asymptotically approach a given value as the frequency of appearances increases, could be used to generate weights in the range of −100 to +100:

  • l n=(200/π)tan−1(s n/3)
  • (The purpose of asymptotically approaching a given value is to steadily decrease the impact of the marginal appearance so that no one keyword overwhelms the others.)
  • Using the data from TABLE I, TABLE II indicates the weights that would be associated with each subordinate keyword by employing the formula above:
  • TABLE II
    n skn sn In
    1 george 2 37
    2 university 2 37
    3 located 1 20
    4 four 1 20
    5 blocks 1 20
    6 white 1 20
    7 house 1 20
    8 created 1 20
    9 act 1 20
    10 congress 1 20
    11 1821 1 20
  • Continuing this example, if the user's second click was on document 256 in FIG. 2C, with the title “George Washington University—Official Athletic Site” and display text “News, schedules, and information about the athletics teams of George Washington University, as well,” the subordinate keywords with their associated weights would look like TABLE III:
  • TABLE III
    n skn sn In
    1 george 4 59
    2 university 4 59
    3 located 1 20
    4 four 1 20
    5 blocks 1 20
    6 white 1 20
    7 house 1 20
    8 created 1 20
    9 act 1 20
    10 congress 1 20
    11 1821 1 20
    12 official 1 20
    13 news 1 20
    14 schedules 1 20
    15 information 1 20
    16 athletics 2 37
    17 teams 1 20
  • Once the subordinate keywords have been generated and each assigned with an appropriate weight, the data is utilized to re-rank the matched documents in the search results. Documents are increased (or decreased) in importance and moved up (or down) in the prioritization of the search results based on their association with the subordinate keywords. With the goal of dynamically re-ranking the search results to best reflect the deduced intent of the user, a ranking algorithm must be developed and then tuned to specify the impact that each subordinate keyword has on the movement of the documents in the search results.
  • The ranking function will run through the search results and adjust the rank of each matched document based on which subordinate keywords are associated with each document, taking into account the weights of each as well as where exactly they are found. The ranking function will, naturally, also take into account the previous rank of the document, helping to reflect, in some way, the intelligence that went into producing the initial order of the search results. Much as the weights of the subordinate keywords were based on where and how often those keywords appeared in relation to the selected, or skipped, documents, the movement of matched documents in the search results will similarly depend on where the subordinate keywords appear. A document with a large number of high-weight subordinate keywords in its title, display text and meta-tags will perhaps move much more dramatically than a document with a few low-weight subordinate keywords buried deep within the text of the document. It should also be noted that the presence of subordinate keywords with positive weights will increase the relevancy of the document, moving it up in the rankings, while the presence of subordinate keywords with negative weights will decrease the relevancy and have the opposite effect.
  • Thus, if M represents the number of matched documents returned by a given query and rm represents the rank of the mth document then the generalized ranking function will look as follows:
  • R ( m ) = f R ( r m ) + n = 1 N f R 1 ( a n , l n ) + f R 2 ( b n , l n ) + f R 3 ( c n , l n ) +
  • Since the actual relevancy scores as determined by the underlying search engine are not necessarily available, the ranks of the matched documents serve as a proxy for relevancy. However, if the underlying search engine were to share the calculated relevancy scores of the matched documents, via some method of communication such as an API, or if the invention was actually incorporated into the underlying search engine itself, then those relevancy scores could be used for rm, in place of the rank, potentially increasing the effectiveness of the system.
  • To illustrate, if tm and dm represent, respectively, the title and display text of the mth document, then TABLE IV is a depiction of the first eight search results as demonstrated in FIG. 2B:
  • TABLE IV
    m rm tm dm
    1 1 Access Washington Home Locate Washington State Government
    Page information and services available on the
    Web.
    2 2 University of Washington Offers information and news for prospective
    and current students, faculty, and staff.
    Highlights . . .
    3 3 Experience Washington The official website of Washington State
    Tourism with maps, history, and tourist
    attractions.
    4 4 Washington DC Convention Official visitor information for Washington,
    and Visitors Association DC - the Nation's Capital, WCTC Home.
    5 5 Washington Post Daily. Offers news, opinion, sports, arts and
    living and entertainment. Includes archives
    since 1977 . . .
    6 6 Washington News - New York Find breaking news, multimedia & opinion
    Times from Washington including news on politics,
    the election, the President, Supreme Court,
    Congress and The White
    7 7 George Washington University The George Washington University is
    located four blocks from the White House
    and was created by an Act of Congress in
    1821.
    8 8 NBA.com: Washington Official site containing news, scores, audio
    Wizards and video files, player statistics and
    schedules.
  • One embodiment of the system might use a ranking function that makes the new rank of a document equal to its previous rank plus some function of the subordinate keywords that appear in the document's title and display text. Where Wr and Ws represent constant weights and Er and Es represent constant exponents, such a ranking function, using the sgn( ) and absolute value functions to handle negative subordinate keyword weights, could be displayed as follows:
  • R ( m ) = W r r m Er + n = 1 N ( number of occurrences of sk n in t m and d m ) W s sgn ( 1 n ) 1 n Es
  • The values of the constants in the ranking function will be developed by careful analysis of empirical user data. The objective is to determine these values in order to optimize the movement of documents and minimize the amount of searching required by the user to find the desired information. One embodiment of the invention uses empirical user data as it is collected to refine the values of the constants in real time. By identifying the end of a successful search, possibly but not necessarily with the help of feedback from the user, the system could, over time, adjust the values of the constants in order to maximize the percentage of searches that end successfully while minimizing the time required to complete a successful search. In one embodiment the constants are actually customized for each user, representing how different users behave differently, and stored in a user profile or cookie. In another embodiment, the constants also depend on other information such as the number of matched documents, which underlying search engine is being used, the language of the results, the country where the user is located, or virtually any other variable.
  • To illustrate how the ranking function works, when the subordinate keywords in TABLE II are applied to the search result documents in TABLE IV, while setting Wr, Er and Es to 1 and Ws to −½, the new rankings, R(m), are produced as displayed in TABLE V:
  • TABLE V
    m rm R(m) tm dm
    1 1 −9 Access Washington Home Locate Washington State Government
    Page information and services available on
    the Web.
    2 2 −16½ University of Washington Offers information and news for
    prospective and current students,
    faculty, and staff. Highlights . . .
    3 3 3 Experience Washington The official website of Washington
    State Tourism with maps, history, and
    tourist attractions.
    4 4 4 Washington DC Convention Official visitor information for
    and Visitors Association Washington, DC - the Nation's
    Capital, WCTC Home.
    5 5 5 Washington Post Daily. Offers news, opinion, sports,
    arts and living and entertainment.
    Includes archives since 1977 . . .
    6 6 −14 Washington News - New Find breaking news, multimedia &
    York Times opinion from Washington including
    news on politics, the election, the
    President, Supreme Court, Congress
    and The White . . .
    7 7 −157 George Washington The George Washington University is
    University located four blocks from the White
    House and was created by an Act of
    Congress in 1821.
    8 8 8 NBA.com: Washington Official site containing news, scores,
    Wizards audio and video files, player statistics
    and schedules.
  • Using the first document (m=1) as an example, the only subordinate keyword from TABLE II found in either the title, t1, or display text, d1, is “locate.” (Techniques, such as stemming, should be employed to, where appropriate, enable the broad matching of terms so that, for example, “located”=“locate.” Artificial intelligence and contextualized matching can also be used to further enhance the term-matching ability of the system.) Since the weight of the subordinate keyword is 20, the ranking equation is thus 1+1×−½×20=−9. Using the second document (m=2) as another example, the only subordinate keyword from TABLE II found in either the title, t2, or display text, d2, is “university,” which has a weight of 37. The ranking equation is thus 2+1×−½×37=−16½. Finally, the seventh document (m=7) represents a more complicated example. The two words “george” and “university,” with weights of 37, both appear twice in the title, t7, and display text, d7. The nine words “located,” “four,” “blocks,” “white,” “house,” “created,” “act,” “congress,” and “1821” all appear once in the display text, d7, and have weights of 20. The ranking function is thus 7+(2×−½×37)×2+(1×−½×20)×9=−157.
  • When sorted by R(m), and then resetting the values of rm for the new rankings, the documents are rearranged as shown in TABLE VI:
  • TABLE VI
    m rm R(m) tm dm
    1 −157 −157 George Washington The George Washington University
    University is located four blocks from the White
    House and was created by an Act of
    Congress in 1821.
    2 −16½ −16½ University of Washington Offers information and news for
    prospective and current students,
    faculty, and staff. Highlights . . .
    3 −14 −14 Washington News - New Find breaking news, multimedia &
    York Times opinion from Washington including
    news on politics, the election, the
    President, Supreme Court, Congress
    and The White . . .
    4 −9 −9 Access Washington Home Locate Washington State
    Page Government information and services
    available on the Web.
    5 3 3 Experience Washington The official website of Washington
    State Tourism with maps, history,
    and tourist attractions.
    6 4 4 Washington DC Convention Official visitor information for
    and Visitors Association Washington, DC - the Nation's
    Capital, WCTC Home.
    7 5 5 Washington Post Daily. Offers news, opinion, sports,
    arts and living and entertainment.
    Includes archives since 1977 . . .
    8 8 8 NBA.com: Washington Official site containing news, scores,
    Wizards audio and video files, player statistics
    and schedules.
  • Some of the new rankings are obviously negative. This does not present a problem, however, since the matched documents are simply ordered from the lowest ranking to the highest. It should also be noted that the calculations in TABLES V and VI will have to be executed on all of the results as opposed to just the first eight, or even just those on the first page of the search results, as it is likely that documents from subsequent pages will be moved forward while others dropped back.
  • As a practical matter, computational limitations imposed by the server hosting the invention software might prohibit running the ranking algorithm on all of the matched documents generated by a query, especially if there are millions of them. Not only is processor speed required to execute all of the calculations, but the server memory might need to be large enough to hold all of the results. Fortunately, the ranking algorithm can be run on a fairly large number of matched documents, the first several hundred or thousand for example, without significantly impacting the effectiveness of the system. Should a determined user page through a large proportion of those re-ranked documents, the system can simply grab the next batch of several hundred or thousand and quickly re-rank those with the previous batch. In any event, it is important to run as many computations as possible in the background while the user is reading or reviewing documents in order to avoid imposing delays on the user.
  • Should the ranking function take into account the presence of subordinate keywords in the actual documents, this could additionally require a significant amount of bandwidth and processing power as each document is downloaded and reviewed. A computational and time-saving technique, however, would be to use the power of the underlying search engine, or even another search engine, to accelerate the speed with which subordinate keywords are identified in matched documents. Rather than scanning all of the matched documents for the presence of subordinate keywords, the system can, in the background, run queries using the subordinate keywords, or just the most important thereof to save on computational time, in order to quickly determine which of the matched documents contain the recently generated subordinate keywords. At this point all that is required is a simple corresponding of the initial matched documents with the results generated by the subordinate keyword queries. The ranking function can then quickly take into account the presence of subordinate keywords in the matched documents themselves and adjust the rankings accordingly.
  • FIG. 5 is a diagram of one embodiment of an “advanced search” page of the invention that actually displays the subordinate keywords to the user. While the matched documents 510-516 and the sponsored links 520-528 are the same, an “information bar” on the left-hand side contains the primary keywords 530 and subordinate keywords 531-550 currently employed. The subordinate keywords can be shown in color, with blue representing positive subordinate keywords and red representing negative subordinate keywords. While the actual color is not important, the intensity of the color corresponds with the weight: the more intense the color, the more the weight is either positive or negative. The subordinate keywords would thus be listed from dark blue (very positive) to dark red (very negative).
  • Displaying the subordinate keywords can be beneficial to the user for a couple of reasons: not only does this give insight into how the system is operating, but users can then assist the system in locating relevant documents by either manually removing or promoting specific subordinate keywords. Should the user know that a particular subordinate keyword is not relevant to the query, that keyword can be selected and then removed by pressing the “remove” button 560. Should the user see a subordinate keyword that is deemed more than just important, but essential, to the query, it can be selected and then promoted to a primary keyword by pressing the “promote” button 562. Once any set of keywords is promoted to primary the initial search will have to be rerun by returning to the underlying search engine for a new set of results. That being said, all of the remaining subordinate keywords and weights can be carried forward with the new set of search results being adjusted accordingly.
  • Since the order of the matched documents changes every time a user returns to the search results page, “bread crumbs” linking to previous search results pages 570-572 will enable the user, if so desired, to go back to previous rankings of matched documents. Additionally, since this system is a novel approach to assisting the user to find relevant documents, the movement of the matched documents might initially be confusing. Nevertheless, a few things can thus be done with the user interface to help ease the transition. “Movement indicators” can be placed immediately to the left of document titles 510-517 to indicate how the documents have moved since the last visit to the search results page (+6, +9, +13, etc.). Also, to further highlight which documents have already been clicked or skipped, boxes of one color, such as blue, can be put around documents that have already been clicked 510 and 512 while boxes of another color, such as red, can be put around documents that have previously been skipped 511 and 515-517. Finally, upon returning to the search results page, the user can be placed at the first document that has yet to be selected or skipped. This will help the user to identify documents that have leapt high in the rankings before continuing with the search process. Other techniques for helping the user understand the process of dynamically changing search results may also be envisioned and implemented.
  • It should be emphasized, as stated earlier, that the sponsored links 520-528 also change dynamically based on the deduced intent of the user. A third-party sponsored link service, that either accepts keyword submissions or scans the content of a page, can take subordinate keywords into account to deliver relevant advertisements. Whether a third-party sponsored link service is used or not, subordinate keywords, since they are a representation of the deduced intent of the user, should be used to dynamically alter the sponsored links that are displayed. To the extent possible, the system should devise techniques for having the selection of sponsored links take the subordinate keywords into consideration. Increasing the accuracy of targeted advertisements that are displayed will have the dual benefit of improving the user experience while increasing the revenue generated by the system.
  • The outcome is thus a dynamic results search engine that continuously changes, updates and reorganizes search results based upon the intent of the user as deduced from the ongoing behavior of the user during the search process. As the user clicks on links, views documents, executes transactions, downloads files, scrolls pages, adds or subtracts keywords (some of which can be taken from generated subordinate keywords), executes other queries, or performs almost any kind of activity, the system takes this information, deduces the intent of the user, generates subordinate keywords and assigns weights to them before then reprioritizing the search results “on the fly.” The end result is a dynamic results search engine better able to assist users in finding desired documents and information.
  • In one embodiment, user behavior monitoring is done by a first software module on server 132 in FIG. 1. The behavior information is provided to a second, re-ranking software module on server 132. The re-ranking module modifies the rankings of search results received from a third-party search engine and stores the re-ranked results on database 130. The re-ranking software module also performs the highlighting, showing of ranking movement, displaying of subordinate keywords, and other functions not performed by the monitoring software module or the third-party search engine. The re-ranked search results are supplied to a client browser on the user's computer. The client browser software may perform some of the customized display functions of the invention.
  • In one embodiment, the re-ranking software module works in parallel with user actions, performing re-ranking while the user behavior is being monitored. A series of re-ranked results can be created and stored in database 130 as the user clicks through documents. If the user decides that a particular document is not relevant, then the re-ranking module takes the subordinate keywords associated with that document, makes their weights negative, and pushes the document (and others that are similar) down.
  • The user browser may be installed on other devices than a computer, such as a personal digital assistant (PDA), a mobile phone, or any other device. The display can be modified to fit a smaller form factor, such as by providing the sponsor links before or after a group of search results. In addition to the visual indicators described for re-ranking, weights of subordinate keywords, etc., audio indications could be used. Additionally, voice input can be used to remove or promote subordinate keywords, or for any other user input.
  • As will be understood by those of skill in the art, the present invention could be embodied in other specific forms without departing from the essential characteristics thereof. For example, in addition to subordinate keywords, data based on prior users' experiences and click streams could be used to re-rank the results in real time. This could provide two levels of ranking, (1) a first re-ranking using subordinate keywords as described above, and (2) a re-ranking of the re-ranked results using the webpages found most desirable by previous users doing similar searches.
  • The present invention can use a separate third party search engine, or could be integrated with a search engine. The search engine could be a general search engine that searches the internet, a specialized search engine that searches a particular web site, a database search engine, a meta-search engine that combines the results of multiple other search engines, or any other type of search engine. Accordingly the foregoing description is intended to be illustrative, but not limiting, of the scope of the invention which is set forth in the following claims.

Claims (19)

1. A method for dynamically modifying search results comprising:
receiving one or more keywords for use as search terms from a user;
providing a first set of search result objects;
monitoring user behavior in interacting with said search result objects; and
generating a second set of search result objects based on said user behavior.
2. The method of claim 1 further comprising:
providing some title and display text with each of said search result objects in said first set;
monitoring which of said objects is selected by said user;
using terms from the title and display text corresponding to a selected object as subordinate keywords for generating said second set of search result objects.
3. The method of claim 2 further comprising:
selecting subordinate keywords from at least one of the links clicked on by the user, other links associated with a document including links pointing to the document, descriptive text associated with each document in the search results, meta-tags connected to viewed documents, prominent words and phrases in viewed documents and a thesaurus.
4. The method of claim 2 wherein said second set of search result objects comprises a re-ranked version of at least a portion of said first set of search result objects.
5. The method of claim 2 further comprising:
using terms from the title and display text corresponding to objects skipped by a user as negative subordinate keywords; and
reducing the ranking of search results objects containing said negative subordinate keywords.
6. The method of claim 2 further comprising:
assigning weights to said subordinate keywords, such that search result objects having higher weighted subordinate keywords are ranked higher.
7. The method of claim 2 further comprising:
selecting and displaying advertising based on said subordinate keywords.
8. The method of claim 2 further comprising:
providing a user interface allowing a user to promote selected subordinate keywords such that they have a higher weighting or are used as primary keywords to generate new search results.
9. The method of claim 2 further comprising:
providing a user interface allowing a user to delete selected subordinate keywords.
10. The method of claim 1 wherein said providing a first set of search result objects is performed by a separate search engine software program from a user behavior program that performs said monitoring and generating steps.
11. The method of claim 1 wherein said monitoring user behavior includes at least one of:
clicks on links in the search results;
subsequent clicks on links within documents;
dwell times;
time spent looking at documents from specific domains;
resources accessed;
transactions conducted;
purchases made;
orders placed;
sessions created;
documents downloaded;
cursors moved;
pages or text scrolled; and
images or other information highlighted.
12. The method of claim 1 further comprising:
returning to said first set of search result objects, after displaying said second set of search result objects, in response to a user indication to return.
13. The method of claim 1 further comprising:
rating the usefulness of at least one of said search result objects in response to a user rating input.
14. The method of claim 1 further comprising:
providing a first display indication of search result objects that have been previously selected by said user; and
providing a second display indication of search result objects that have been skipped by said user.
15. The method of claim 1 further comprising:
providing a display indication of the ranking movement of said search result objects in said second set of search result objects.
16. A system for dynamically modifying search results comprising:
a user interface on a first computer configured to receive one or more keywords for use as search terms from a user;
a search engine on a server in communication with said first computer over a network, said search engine being configured to provide a first set of search result objects in response to said keywords;
a monitoring program in communication with said network, said monitoring program being configured to monitor user behavior in interacting with said search result objects; and
a re-ranking program configured to generate a second set of search result objects based on said user behavior.
17. The system of claim 16 wherein said search engine is on a first server, and said monitoring and re-ranking programs are on a second server.
18. Computer readable media having computer readable code thereon for dynamically modifying search results, comprising code for:
receiving one or more keywords for use as search terms from a user;
providing a first set of search result objects;
monitoring user behavior in interacting with said search result objects; and
generating a second set of search result objects based on said user behavior.
19. A method for ranking a set of search results having an initial ranking, comprising:
determining one or more selected objects in said set of search results which corresponds to user interest based on user behavior;
selecting at least one subordinate keyword corresponding to said selected objects; and
re-ranking said set of search results based on said at least one subordinate keyword.
US11/510,524 2006-05-02 2006-08-25 Dynamic search engine results employing user behavior Abandoned US20070260597A1 (en)

Priority Applications (8)

Application Number Priority Date Filing Date Title
US11/510,524 US20070260597A1 (en) 2006-05-02 2006-08-25 Dynamic search engine results employing user behavior
PCT/US2007/067799 WO2007130914A2 (en) 2006-05-02 2007-04-30 Dynamic search engine results employing user behavior
US12/652,004 US8095582B2 (en) 2006-05-02 2010-01-04 Dynamic search engine results employing user behavior
US13/315,199 US20120078710A1 (en) 2006-05-02 2011-12-08 Dynamic search engine results employing user behavior
US14/018,791 US8924378B2 (en) 2006-08-25 2013-09-05 Adaptive user interface for real-time search relevance feedback
US14/552,320 US9418122B2 (en) 2006-08-25 2014-11-24 Adaptive user interface for real-time search relevance feedback
US15/053,834 US20160350428A1 (en) 2006-05-02 2016-02-25 Real time implicit user modeling for personalized search
US15/207,277 US20170039285A1 (en) 2006-08-25 2016-07-11 Adaptive user interface for real-time search relevance feedback

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US79662406P 2006-05-02 2006-05-02
US11/510,524 US20070260597A1 (en) 2006-05-02 2006-08-25 Dynamic search engine results employing user behavior

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US12/652,004 Continuation US8095582B2 (en) 2006-05-02 2010-01-04 Dynamic search engine results employing user behavior

Publications (1)

Publication Number Publication Date
US20070260597A1 true US20070260597A1 (en) 2007-11-08

Family

ID=38662295

Family Applications (3)

Application Number Title Priority Date Filing Date
US11/510,524 Abandoned US20070260597A1 (en) 2006-05-02 2006-08-25 Dynamic search engine results employing user behavior
US12/652,004 Active US8095582B2 (en) 2006-05-02 2010-01-04 Dynamic search engine results employing user behavior
US13/315,199 Abandoned US20120078710A1 (en) 2006-05-02 2011-12-08 Dynamic search engine results employing user behavior

Family Applications After (2)

Application Number Title Priority Date Filing Date
US12/652,004 Active US8095582B2 (en) 2006-05-02 2010-01-04 Dynamic search engine results employing user behavior
US13/315,199 Abandoned US20120078710A1 (en) 2006-05-02 2011-12-08 Dynamic search engine results employing user behavior

Country Status (2)

Country Link
US (3) US20070260597A1 (en)
WO (1) WO2007130914A2 (en)

Cited By (124)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070239680A1 (en) * 2006-03-30 2007-10-11 Oztekin Bilgehan U Website flavored search
US20070271255A1 (en) * 2006-05-17 2007-11-22 Nicky Pappo Reverse search-engine
US20080082477A1 (en) * 2006-09-29 2008-04-03 Microsoft Corporation Key phrase extraction from query logs
US20080168045A1 (en) * 2007-01-10 2008-07-10 Microsoft Corporation Content rank
US20080244428A1 (en) * 2007-03-30 2008-10-02 Yahoo! Inc. Visually Emphasizing Query Results Based on Relevance Feedback
US20080263025A1 (en) * 2007-04-20 2008-10-23 Koran Joshua M Use of natural search click events to augment online campaigns
US20080270473A1 (en) * 2007-04-30 2008-10-30 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Determining an influence on a person by web pages
US20090089311A1 (en) * 2007-09-28 2009-04-02 Yahoo! Inc. System and method for inclusion of history in a search results page
US20090119278A1 (en) * 2007-11-07 2009-05-07 Cross Tiffany B Continual Reorganization of Ordered Search Results Based on Current User Interaction
US20090119254A1 (en) * 2007-11-07 2009-05-07 Cross Tiffany B Storing Accessible Histories of Search Results Reordered to Reflect User Interest in the Search Results
US20090158146A1 (en) * 2007-12-13 2009-06-18 Concert Technology Corporation Resizing tag representations or tag group representations to control relative importance
US20090259647A1 (en) * 2008-04-10 2009-10-15 Concert Technology Corporation Fuzzy keyword searching
US20090282021A1 (en) * 2008-05-12 2009-11-12 Bennett James D Web browser accessible search engine which adapts based on user interaction
US20090292685A1 (en) * 2008-05-22 2009-11-26 Microsoft Corporation Video search re-ranking via multi-graph propagation
US20090319514A1 (en) * 2008-03-26 2009-12-24 Thales Method and system for assigning scores
US20100017392A1 (en) * 2008-07-18 2010-01-21 Jianwei Dian Intent match search engine
US20100057714A1 (en) * 2008-09-04 2010-03-04 John Edward Miedema Search results ranking method and system
US20100082610A1 (en) * 2008-09-30 2010-04-01 Peter Anick Query refinement based on user selections
US20100082566A1 (en) * 2008-10-01 2010-04-01 Microsoft Corporation Evaluating the ranking quality of a ranked list
US20100131835A1 (en) * 2008-11-22 2010-05-27 Srihari Kumar System and methods for inferring intent of website visitors and generating and packaging visitor information for distribution as sales leads or market intelligence
US20100199219A1 (en) * 2008-12-31 2010-08-05 Robert Poniatowski Adaptive search result user interface
US20100208984A1 (en) * 2009-02-13 2010-08-19 Microsoft Corporation Evaluating related phrases
US20100241624A1 (en) * 2009-03-20 2010-09-23 Microsoft Corporation Presenting search results ordered using user preferences
US7827170B1 (en) 2007-03-13 2010-11-02 Google Inc. Systems and methods for demoting personalized search results based on personal information
US20110022578A1 (en) * 2009-07-24 2011-01-27 Krassimir Fotev System and method for ranking documents through human assistance
US20110179453A1 (en) * 2008-12-31 2011-07-21 Poniatowski Robert F Methods and techniques for adaptive search
US20110231396A1 (en) * 2010-03-19 2011-09-22 Avaya Inc. System and method for providing predictive contacts
WO2011116301A2 (en) * 2010-03-19 2011-09-22 Visa U.S.A. Inc. Systems and methods to distribute advertisement opportunities to merchants
WO2011116296A2 (en) * 2010-03-19 2011-09-22 Visa U.S.A. Inc. Systems and methods to enhance search data with transaction based data
US20110238659A1 (en) * 2010-03-29 2011-09-29 Ebay Inc. Two-pass searching for image similarity of digests of image-based listings in a network-based publication system
US20110235902A1 (en) * 2010-03-29 2011-09-29 Ebay Inc. Pre-computing digests for image similarity searching of image-based listings in a network-based publication system
WO2011130019A2 (en) 2010-04-14 2011-10-20 Microsoft Corporation Search advertisement selection based on user actions
US8082246B2 (en) 2004-09-30 2011-12-20 Microsoft Corporation System and method for ranking search results using click distance
US20120143790A1 (en) * 2010-12-01 2012-06-07 Microsoft Corporation Relevance of search results determined from user clicks and post-click user behavior obtained from click logs
KR20120089560A (en) * 2008-09-10 2012-08-13 마이크로소프트 코포레이션 Document length as a static relevance feature for ranking search results
US8380705B2 (en) 2003-09-12 2013-02-19 Google Inc. Methods and systems for improving a search ranking using related queries
US8396865B1 (en) 2008-12-10 2013-03-12 Google Inc. Sharing search engine relevance data between corpora
US20130086083A1 (en) * 2011-09-30 2013-04-04 Microsoft Corporation Transferring ranking signals from equivalent pages
US20130124512A1 (en) * 2006-10-11 2013-05-16 Collarity, Inc. Negative associations for generation of refinement options
US20130138637A1 (en) * 2009-09-21 2013-05-30 Walter Bachtiger Systems and methods for ranking media files
CN103164521A (en) * 2013-03-11 2013-06-19 亿赞普(北京)科技有限公司 Keyword calculation method and device based on user browse and search actions
US8484199B1 (en) * 2006-12-12 2013-07-09 Google Inc. Ranking of geographic information
US8498974B1 (en) 2009-08-31 2013-07-30 Google Inc. Refining search results
US8543570B1 (en) * 2008-06-10 2013-09-24 Surf Canyon Incorporated Adaptive user interface for real-time search relevance feedback
US20130254802A1 (en) * 2007-06-27 2013-09-26 Google Inc. Selection of advertisements for placement with content
US20130254031A1 (en) * 2006-12-12 2013-09-26 International Business Machines Corporation Dynamic Modification of Advertisements Displayed in Response to a Search Engine Query
US8612460B1 (en) * 2007-03-27 2013-12-17 Google Inc. Content entity management
US8615514B1 (en) 2010-02-03 2013-12-24 Google Inc. Evaluating website properties by partitioning user feedback
US8639567B2 (en) 2010-03-19 2014-01-28 Visa U.S.A. Inc. Systems and methods to identify differences in spending patterns
US8645817B1 (en) 2006-12-29 2014-02-04 Monster Worldwide, Inc. Apparatuses, methods and systems for enhanced posted listing generation and distribution management
US20140040754A1 (en) * 2009-08-27 2014-02-06 Apple Inc. Adaptive mapping of search results
US8661029B1 (en) 2006-11-02 2014-02-25 Google Inc. Modifying search result ranking based on implicit user feedback
US8694511B1 (en) 2007-08-20 2014-04-08 Google Inc. Modifying search result ranking based on populations
US8694374B1 (en) 2007-03-14 2014-04-08 Google Inc. Detecting click spam
USD703687S1 (en) 2011-12-28 2014-04-29 Target Brands, Inc. Display screen with graphical user interface
USD703686S1 (en) 2011-12-28 2014-04-29 Target Brands, Inc. Display screen with graphical user interface
USD703685S1 (en) 2011-12-28 2014-04-29 Target Brands, Inc. Display screen with graphical user interface
US8713003B2 (en) 2009-07-24 2014-04-29 Peer Belt Inc. System and method for ranking content and applications through human assistance
US20140122475A1 (en) * 2012-10-29 2014-05-01 Alibaba Group Holding Limited Search result ranking method and system
US8732151B2 (en) 2011-04-01 2014-05-20 Microsoft Corporation Enhanced query rewriting through statistical machine translation
US8738635B2 (en) 2010-06-01 2014-05-27 Microsoft Corporation Detection of junk in search result ranking
USD705792S1 (en) * 2011-12-28 2014-05-27 Target Brands, Inc. Display screen with graphical user interface
USD705790S1 (en) 2011-12-28 2014-05-27 Target Brands, Inc. Display screen with graphical user interface
USD706794S1 (en) 2011-12-28 2014-06-10 Target Brands, Inc. Display screen with graphical user interface
USD706793S1 (en) 2011-12-28 2014-06-10 Target Brands, Inc. Display screen with graphical user interface
US8762373B1 (en) 2006-09-29 2014-06-24 Google Inc. Personalized search result ranking
US20140229812A1 (en) * 2008-04-11 2014-08-14 Advertising.Com Llc Systems and methods for video content association
USD711400S1 (en) 2011-12-28 2014-08-19 Target Brands, Inc. Display screen with graphical user interface
US8812493B2 (en) 2008-04-11 2014-08-19 Microsoft Corporation Search results ranking using editing distance and document information
USD711399S1 (en) 2011-12-28 2014-08-19 Target Brands, Inc. Display screen with graphical user interface
US8832083B1 (en) 2010-07-23 2014-09-09 Google Inc. Combining user feedback
US8843486B2 (en) 2004-09-27 2014-09-23 Microsoft Corporation System and method for scoping searches using index keys
USD715818S1 (en) 2011-12-28 2014-10-21 Target Brands, Inc. Display screen with graphical user interface
US8874570B1 (en) 2004-11-30 2014-10-28 Google Inc. Search boost vector based on co-visitation information
US8898153B1 (en) 2009-11-20 2014-11-25 Google Inc. Modifying scoring data based on historical changes
US8909655B1 (en) 2007-10-11 2014-12-09 Google Inc. Time based ranking
US8924379B1 (en) 2010-03-05 2014-12-30 Google Inc. Temporal-based score adjustments
US8938463B1 (en) 2007-03-12 2015-01-20 Google Inc. Modifying search result ranking based on implicit user feedback and a model of presentation bias
US8949252B2 (en) 2010-03-29 2015-02-03 Ebay Inc. Product category optimization for image similarity searching of image-based listings in a network-based publication system
US8954463B2 (en) * 2012-02-29 2015-02-10 International Business Machines Corporation Use of statistical language modeling for generating exploratory search results
US8959093B1 (en) 2010-03-15 2015-02-17 Google Inc. Ranking search results based on anchors
US8972391B1 (en) 2009-10-02 2015-03-03 Google Inc. Recent interest based relevance scoring
US8972394B1 (en) 2009-07-20 2015-03-03 Google Inc. Generating a related set of documents for an initial set of documents
US20150066973A1 (en) * 2013-08-29 2015-03-05 Google Inc. Modifying Search Results Based on Dismissal Action Associated With One or More of the Search Results
US9002867B1 (en) 2010-12-30 2015-04-07 Google Inc. Modifying ranking data based on document changes
US9009146B1 (en) 2009-04-08 2015-04-14 Google Inc. Ranking search results based on similar queries
US20150112984A1 (en) * 2012-07-13 2015-04-23 Sony Computer Entertainment Inc. Processing apparatus
US9092510B1 (en) 2007-04-30 2015-07-28 Google Inc. Modifying search result ranking based on a temporal element of user feedback
US9110975B1 (en) * 2006-11-02 2015-08-18 Google Inc. Search result inputs using variant generalized queries
US9183499B1 (en) 2013-04-19 2015-11-10 Google Inc. Evaluating quality based on neighbor features
US20150324848A1 (en) * 2004-10-01 2015-11-12 Ricoh Co., Ltd. Dynamic Presentation of Targeted Information in a Mixed Media Reality Recognition System
CN105069102A (en) * 2015-08-03 2015-11-18 百度在线网络技术(北京)有限公司 Information push method and apparatus
US9495462B2 (en) 2012-01-27 2016-11-15 Microsoft Technology Licensing, Llc Re-ranking search results
US9529919B2 (en) 2010-03-29 2016-12-27 Paypal, Inc. Traffic driver for suggesting stores
US9623119B1 (en) 2010-06-29 2017-04-18 Google Inc. Accentuating search results
US9762528B2 (en) 2011-07-27 2017-09-12 Ricoh Co., Ltd. Generating a conversation in a social network based on mixed media object context
US9779390B1 (en) 2008-04-21 2017-10-03 Monster Worldwide, Inc. Apparatuses, methods and systems for advancement path benchmarking
US9781202B2 (en) 2010-01-19 2017-10-03 Collarity, Inc. Anchoring for content synchronization
US9870388B2 (en) 2006-07-31 2018-01-16 Ricoh, Co., Ltd. Analyzing usage of visual content to determine relationships indicating unsuccessful attempts to retrieve the visual content
US9959525B2 (en) 2005-05-23 2018-05-01 Monster Worldwide, Inc. Intelligent job matching system and method
US9972108B2 (en) 2006-07-31 2018-05-15 Ricoh Co., Ltd. Mixed media reality recognition with image tracking
US20180152431A1 (en) * 2016-11-28 2018-05-31 Yandex Europe Ag Method of selecting a website for displaying in a web browser quick-access field
CN108108380A (en) * 2016-11-25 2018-06-01 阿里巴巴集团控股有限公司 Search ordering method, searching order device, searching method and searcher
US10073859B2 (en) 2004-10-01 2018-09-11 Ricoh Co., Ltd. System and methods for creation and use of a mixed media environment
WO2019007985A1 (en) * 2017-07-05 2019-01-10 Technische Universität Darmstadt Method and device for flexible ranking system for information in a multi-linked network
US10181116B1 (en) 2006-01-09 2019-01-15 Monster Worldwide, Inc. Apparatuses, systems and methods for data entry correlation
US10192279B1 (en) 2007-07-11 2019-01-29 Ricoh Co., Ltd. Indexed document modification sharing with mixed media reality
US10282358B2 (en) 2015-09-30 2019-05-07 Yandex Europe Ag Methods of furnishing search results to a plurality of client devices via a search engine system
US10360605B2 (en) * 2010-03-29 2019-07-23 Rakuten, Inc. Server apparatus, information providing method, information providing program, recording medium recording the information providing program, and information providing system
US10387839B2 (en) 2006-03-31 2019-08-20 Monster Worldwide, Inc. Apparatuses, methods and systems for automated online data submission
US10387559B1 (en) * 2016-11-22 2019-08-20 Google Llc Template-based identification of user interest
US10496691B1 (en) 2015-09-08 2019-12-03 Google Llc Clustering search results
CN110781383A (en) * 2019-09-27 2020-02-11 上海麦克风文化传媒有限公司 Method for determining comprehensive sequencing investigation index
US10607150B2 (en) 2016-02-23 2020-03-31 Splunk Inc. Machine-learning data analysis tool
US10679142B2 (en) * 2016-04-29 2020-06-09 Splunk Inc. Dynamic search guidance for machine data indexing and search system
US10789417B1 (en) * 2012-05-24 2020-09-29 The Boeing Company Method and apparatus for identifying relationships between documents
US10896461B2 (en) * 2014-06-25 2021-01-19 Beijing Baidu Netcom Science And Technology Co., Ltd. Method and apparatus for data mining based on users' search behavior
CN112597361A (en) * 2020-12-16 2021-04-02 北京五八信息技术有限公司 Sorting processing method and device, electronic equipment and storage medium
US11157576B2 (en) * 2013-07-31 2021-10-26 Tencent Technology (Shenzhen) Company Limited Method, system and terminal for performing search in a browser
US11295374B2 (en) 2010-08-28 2022-04-05 Ebay Inc. Multilevel silhouettes in an online shopping environment
US20220414167A1 (en) * 2019-04-30 2022-12-29 Capital One Services, Llc Techniques to leverage machine learning for search engine optimization
US11605116B2 (en) 2010-03-29 2023-03-14 Ebay Inc. Methods and systems for reducing item selection error in an e-commerce environment
US11887184B2 (en) 2019-05-02 2024-01-30 Capital One Services, Llc Techniques to facilitate online commerce by leveraging user activity
US11935103B2 (en) 2021-12-29 2024-03-19 Ebay Inc. Methods and systems for reducing item selection error in an e-commerce environment

Families Citing this family (77)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8280877B2 (en) * 2007-02-22 2012-10-02 Microsoft Corporation Diverse topic phrase extraction
US8019742B1 (en) 2007-05-31 2011-09-13 Google Inc. Identifying related queries
US9392074B2 (en) 2007-07-07 2016-07-12 Qualcomm Incorporated User profile generation architecture for mobile content-message targeting
US20090048977A1 (en) * 2007-07-07 2009-02-19 Qualcomm Incorporated User profile generation architecture for targeted content distribution using external processes
US20090013051A1 (en) 2007-07-07 2009-01-08 Qualcomm Incorporated Method for transfer of information related to targeted content messages through a proxy server
US9705998B2 (en) * 2007-11-14 2017-07-11 Qualcomm Incorporated Method and system using keyword vectors and associated metrics for learning and prediction of user correlation of targeted content messages in a mobile environment
KR101195630B1 (en) * 2007-11-14 2012-10-31 콸콤 인코포레이티드 Methods and systems for determining a geographic user profile to determine suitability of targeted content messages based on the profile
US20090177530A1 (en) * 2007-12-14 2009-07-09 Qualcomm Incorporated Near field communication transactions in a mobile environment
US7908262B2 (en) * 2008-02-19 2011-03-15 Surfjar, Inc. System and method for providing search engine-based rewards
US9710817B2 (en) * 2008-09-30 2017-07-18 Microsoft Technology Licensing, Llc Adaptive run-time advertisements
US20090327520A1 (en) * 2008-06-26 2009-12-31 Felix Lee Method and apparatus for mapping unknown information in real or virtual worlds
US9183323B1 (en) 2008-06-27 2015-11-10 Google Inc. Suggesting alternative query phrases in query results
JP2010026724A (en) * 2008-07-17 2010-02-04 Fujitsu Ltd Web page providing apparatus, method for interlocking web page with ranking and program thereof
US9330165B2 (en) 2009-02-13 2016-05-03 Microsoft Technology Licensing, Llc Context-aware query suggestion by mining log data
US8019746B2 (en) * 2009-03-02 2011-09-13 Yahoo! Inc. Optimized search result columns on search results pages
US9247012B2 (en) 2009-12-23 2016-01-26 International Business Machines Corporation Applying relative weighting schemas to online usage data
US8849785B1 (en) * 2010-01-15 2014-09-30 Google Inc. Search query reformulation using result term occurrence count
US20110191327A1 (en) * 2010-01-31 2011-08-04 Advanced Research Llc Method for Human Ranking of Search Results
US8924376B1 (en) * 2010-01-31 2014-12-30 Bryant Christopher Lee Method for human ranking of search results
US20110208730A1 (en) * 2010-02-23 2011-08-25 Microsoft Corporation Context-aware searching
US20110282869A1 (en) * 2010-05-11 2011-11-17 Maxim Zhilyaev Access to information by quantitative analysis of enterprise web access traffic
US8600979B2 (en) 2010-06-28 2013-12-03 Yahoo! Inc. Infinite browse
US20110320424A1 (en) * 2010-06-29 2011-12-29 Intuit Inc. Assessing and adapting component parameters
US20120005186A1 (en) * 2010-06-30 2012-01-05 Dante Monteverde System and method for aggregating and interactive ranking of search engine results
US8868548B2 (en) * 2010-07-22 2014-10-21 Google Inc. Determining user intent from query patterns
CN103080963A (en) * 2010-08-11 2013-05-01 瑞典爱立信有限公司 Face-directional recognition driven display controL
US20120158953A1 (en) * 2010-12-21 2012-06-21 Raytheon Bbn Technologies Corp. Systems and methods for monitoring and mitigating information leaks
US8799297B2 (en) * 2011-03-21 2014-08-05 Aol Inc. Evaluating supply of electronic content relating to keywords
US11841912B2 (en) 2011-05-01 2023-12-12 Twittle Search Limited Liability Company System for applying natural language processing and inputs of a group of users to infer commonly desired search results
US8326862B2 (en) * 2011-05-01 2012-12-04 Alan Mark Reznik Systems and methods for facilitating enhancements to search engine results
US9268857B2 (en) 2011-06-03 2016-02-23 Facebook, Inc. Suggesting search results to users before receiving any search query from the users
US9110992B2 (en) 2011-06-03 2015-08-18 Facebook, Inc. Context-based selection of calls-to-action associated with search results
US8909637B2 (en) 2011-06-03 2014-12-09 Facebook, Inc. Context-based ranking of search results
US9519726B2 (en) * 2011-06-16 2016-12-13 Amit Kumar Surfacing applications based on browsing activity
US9547693B1 (en) * 2011-06-23 2017-01-17 Palantir Technologies Inc. Periodic database search manager for multiple data sources
US8645300B1 (en) * 2011-07-20 2014-02-04 Usability Sciences Corporation System and method for intent data processing
CA2844065C (en) 2011-08-04 2018-04-03 Google Inc. Providing knowledge panels with search results
WO2013025536A1 (en) * 2011-08-12 2013-02-21 Ebay Inc. Systems and methods for personalized pricing
US20130054355A1 (en) * 2011-08-29 2013-02-28 Microsoft Corporation Advertisement customization
US9053087B2 (en) * 2011-09-23 2015-06-09 Microsoft Technology Licensing, Llc Automatic semantic evaluation of speech recognition results
US9274683B2 (en) 2011-12-30 2016-03-01 Google Inc. Interactive answer boxes for user search queries
US20150169643A1 (en) * 2012-05-14 2015-06-18 Google Inc. Providing supplemental search results in repsonse to user interest signal
US9098586B1 (en) 2012-07-05 2015-08-04 EarthNetTV Inc. Private user interest recognition and targeted recommendation system
US9779140B2 (en) 2012-11-16 2017-10-03 Google Inc. Ranking signals for sparse corpora
US10108720B2 (en) * 2012-11-28 2018-10-23 International Business Machines Corporation Automatically providing relevant search results based on user behavior
US9542491B2 (en) 2012-12-14 2017-01-10 Microsoft Technology Licensing, Llc Utilizing keystroke logging to determine items for presentation
US10192238B2 (en) * 2012-12-21 2019-01-29 Walmart Apollo, Llc Real-time bidding and advertising content generation
US9086825B2 (en) * 2013-01-23 2015-07-21 Hewlett-Packard Development Company, L.P. Providing supplemental content based on a selected file
US9019548B2 (en) 2013-01-24 2015-04-28 Hewlett-Packard Development Company, L.P. Print intent type
US9256341B2 (en) 2013-03-20 2016-02-09 Microsoft Technology Licensing, Llc Tracking changes in collaborative authoring environment
US9116975B2 (en) 2013-10-18 2015-08-25 Palantir Technologies Inc. Systems and user interfaces for dynamic and interactive simultaneous querying of multiple data stores
US9569536B2 (en) 2013-12-17 2017-02-14 Microsoft Technology Licensing, Llc Identifying similar applications
US10824787B2 (en) 2013-12-21 2020-11-03 Microsoft Technology Licensing, Llc Authoring through crowdsourcing based suggestions
US20150178391A1 (en) * 2013-12-21 2015-06-25 Microsoft Technology Licensing, Llc Intent based content related suggestions as small multiples
US11514399B2 (en) 2013-12-21 2022-11-29 Microsoft Technology Licensing, Llc Authoring through suggestion
CN104933081B (en) * 2014-03-21 2018-06-29 阿里巴巴集团控股有限公司 Providing method and device are suggested in a kind of search
US10528980B2 (en) * 2014-06-09 2020-01-07 Adobe Inc. Targeted marketing using dynamically varied recommendations
US9535974B1 (en) 2014-06-30 2017-01-03 Palantir Technologies Inc. Systems and methods for identifying key phrase clusters within documents
US9619557B2 (en) 2014-06-30 2017-04-11 Palantir Technologies, Inc. Systems and methods for key phrase characterization of documents
US10320633B1 (en) * 2014-11-20 2019-06-11 BloomReach Inc. Insights for web service providers
US10423704B2 (en) * 2014-12-17 2019-09-24 International Business Machines Corporation Utilizing hyperlink forward chain analysis to signify relevant links to a user
US9348920B1 (en) 2014-12-22 2016-05-24 Palantir Technologies Inc. Concept indexing among database of documents using machine learning techniques
US10552994B2 (en) 2014-12-22 2020-02-04 Palantir Technologies Inc. Systems and interactive user interfaces for dynamic retrieval, analysis, and triage of data items
US9817563B1 (en) 2014-12-29 2017-11-14 Palantir Technologies Inc. System and method of generating data points from one or more data stores of data items for chart creation and manipulation
CN106156109B (en) * 2015-04-03 2020-09-04 阿里巴巴集团控股有限公司 Searching method and device
US11137870B2 (en) * 2015-08-11 2021-10-05 Ebay Inc. Adjusting an interface based on a cognitive mode
US11449906B1 (en) * 2015-09-30 2022-09-20 Groupon, Inc. Dynamic augmenting relevance rankings using data from external ratings sources
US10296623B2 (en) 2015-11-03 2019-05-21 International Business Machines Corporation Document curation
CN106997360A (en) * 2016-01-25 2017-08-01 阿里巴巴集团控股有限公司 The treating method and apparatus of user behavior data
US20180018333A1 (en) * 2016-07-18 2018-01-18 Bioz, Inc. Continuous evaluation and adjustment of search engine results
US10318630B1 (en) 2016-11-21 2019-06-11 Palantir Technologies Inc. Analysis of large bodies of textual data
US20180150525A1 (en) * 2016-11-29 2018-05-31 Facebook, Inc. Methods and Systems for Prioritizing Entities in Search Results
WO2019234736A1 (en) * 2018-06-06 2019-12-12 Comvert Ltd. Systems and methods for dynamic adaptation of a graphical user interface
WO2020092670A2 (en) 2018-11-02 2020-05-07 Walmart Apollo, Llc Systems and methods for search modification
US11284171B1 (en) * 2020-02-20 2022-03-22 Amazon Technologies, Inc. Automated and guided video content exploration and discovery
US20210406993A1 (en) * 2020-06-29 2021-12-30 Dell Products L.P. Automated generation of titles and descriptions for electronic commerce products
US11586841B2 (en) 2020-07-01 2023-02-21 Wipro Limited Method and system for generating user driven adaptive object visualizations using generative adversarial network models

Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5806077A (en) * 1993-01-15 1998-09-08 International Business Machines Corporation Hypertext display system
US5890152A (en) * 1996-09-09 1999-03-30 Seymour Alvin Rapaport Personal feedback browser for obtaining media files
US5946678A (en) * 1995-01-11 1999-08-31 Philips Electronics North America Corporation User interface for document retrieval
US6134532A (en) * 1997-11-14 2000-10-17 Aptex Software, Inc. System and method for optimal adaptive matching of users to most relevant entity and information in real-time
US6370526B1 (en) * 1999-05-18 2002-04-09 International Business Machines Corporation Self-adaptive method and system for providing a user-preferred ranking order of object sets
US20020107776A1 (en) * 2001-02-05 2002-08-08 Homestore.Com System and method for anonymous lead generation and management
US20030088463A1 (en) * 1999-10-21 2003-05-08 Steven Fischman System and method for group advertisement optimization
US20030135490A1 (en) * 2002-01-15 2003-07-17 Barrett Michael E. Enhanced popularity ranking
US20030195877A1 (en) * 1999-12-08 2003-10-16 Ford James L. Search query processing to provide category-ranked presentation of search results
US6691106B1 (en) * 2000-05-23 2004-02-10 Intel Corporation Profile driven instant web portal
US6738764B2 (en) * 2001-05-08 2004-05-18 Verity, Inc. Apparatus and method for adaptively ranking search results
US20040111678A1 (en) * 2002-10-01 2004-06-10 Masaaki Hara Method for retrieving documents
US20050060311A1 (en) * 2003-09-12 2005-03-17 Simon Tong Methods and systems for improving a search ranking using related queries
US20050071328A1 (en) * 2003-09-30 2005-03-31 Lawrence Stephen R. Personalization of web search
US20050165781A1 (en) * 2004-01-26 2005-07-28 Reiner Kraft Method, system, and program for handling anchor text
US20050192863A1 (en) * 2004-02-26 2005-09-01 Krishna Mohan Web site vistor incentive program in conjunction with promotion of anonymously identifying a user and/or a group
US20060064411A1 (en) * 2004-09-22 2006-03-23 William Gross Search engine using user intent
US20060136589A1 (en) * 1999-12-28 2006-06-22 Utopy, Inc. Automatic, personalized online information and product services
US7475072B1 (en) * 2005-09-26 2009-01-06 Quintura, Inc. Context-based search visualization and context management using neural networks

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050144158A1 (en) * 2003-11-18 2005-06-30 Capper Liesl J. Computer network search engine
US7634474B2 (en) * 2006-03-30 2009-12-15 Microsoft Corporation Using connectivity distance for relevance feedback in search

Patent Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5806077A (en) * 1993-01-15 1998-09-08 International Business Machines Corporation Hypertext display system
US5946678A (en) * 1995-01-11 1999-08-31 Philips Electronics North America Corporation User interface for document retrieval
US5890152A (en) * 1996-09-09 1999-03-30 Seymour Alvin Rapaport Personal feedback browser for obtaining media files
US6134532A (en) * 1997-11-14 2000-10-17 Aptex Software, Inc. System and method for optimal adaptive matching of users to most relevant entity and information in real-time
US6370526B1 (en) * 1999-05-18 2002-04-09 International Business Machines Corporation Self-adaptive method and system for providing a user-preferred ranking order of object sets
US20030088463A1 (en) * 1999-10-21 2003-05-08 Steven Fischman System and method for group advertisement optimization
US20030195877A1 (en) * 1999-12-08 2003-10-16 Ford James L. Search query processing to provide category-ranked presentation of search results
US20060136589A1 (en) * 1999-12-28 2006-06-22 Utopy, Inc. Automatic, personalized online information and product services
US6691106B1 (en) * 2000-05-23 2004-02-10 Intel Corporation Profile driven instant web portal
US20020107776A1 (en) * 2001-02-05 2002-08-08 Homestore.Com System and method for anonymous lead generation and management
US6738764B2 (en) * 2001-05-08 2004-05-18 Verity, Inc. Apparatus and method for adaptively ranking search results
US20030135490A1 (en) * 2002-01-15 2003-07-17 Barrett Michael E. Enhanced popularity ranking
US20040111678A1 (en) * 2002-10-01 2004-06-10 Masaaki Hara Method for retrieving documents
US20050060311A1 (en) * 2003-09-12 2005-03-17 Simon Tong Methods and systems for improving a search ranking using related queries
US20050071328A1 (en) * 2003-09-30 2005-03-31 Lawrence Stephen R. Personalization of web search
US20050165781A1 (en) * 2004-01-26 2005-07-28 Reiner Kraft Method, system, and program for handling anchor text
US20050192863A1 (en) * 2004-02-26 2005-09-01 Krishna Mohan Web site vistor incentive program in conjunction with promotion of anonymously identifying a user and/or a group
US20060064411A1 (en) * 2004-09-22 2006-03-23 William Gross Search engine using user intent
US7475072B1 (en) * 2005-09-26 2009-01-06 Quintura, Inc. Context-based search visualization and context management using neural networks

Cited By (206)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8452758B2 (en) 2003-09-12 2013-05-28 Google Inc. Methods and systems for improving a search ranking using related queries
US8380705B2 (en) 2003-09-12 2013-02-19 Google Inc. Methods and systems for improving a search ranking using related queries
US9542453B1 (en) 2004-07-13 2017-01-10 Google Inc. Systems and methods for promoting search results based on personal information
US8843486B2 (en) 2004-09-27 2014-09-23 Microsoft Corporation System and method for scoping searches using index keys
US8082246B2 (en) 2004-09-30 2011-12-20 Microsoft Corporation System and method for ranking search results using click distance
US20150324848A1 (en) * 2004-10-01 2015-11-12 Ricoh Co., Ltd. Dynamic Presentation of Targeted Information in a Mixed Media Reality Recognition System
US10007928B2 (en) * 2004-10-01 2018-06-26 Ricoh Company, Ltd. Dynamic presentation of targeted information in a mixed media reality recognition system
US10073859B2 (en) 2004-10-01 2018-09-11 Ricoh Co., Ltd. System and methods for creation and use of a mixed media environment
US8874570B1 (en) 2004-11-30 2014-10-28 Google Inc. Search boost vector based on co-visitation information
US9959525B2 (en) 2005-05-23 2018-05-01 Monster Worldwide, Inc. Intelligent job matching system and method
US10181116B1 (en) 2006-01-09 2019-01-15 Monster Worldwide, Inc. Apparatuses, systems and methods for data entry correlation
US20070239680A1 (en) * 2006-03-30 2007-10-11 Oztekin Bilgehan U Website flavored search
US8078607B2 (en) 2006-03-30 2011-12-13 Google Inc. Generating website profiles based on queries from webistes and user activities on the search results
US10387839B2 (en) 2006-03-31 2019-08-20 Monster Worldwide, Inc. Apparatuses, methods and systems for automated online data submission
US20070271255A1 (en) * 2006-05-17 2007-11-22 Nicky Pappo Reverse search-engine
US9972108B2 (en) 2006-07-31 2018-05-15 Ricoh Co., Ltd. Mixed media reality recognition with image tracking
US9870388B2 (en) 2006-07-31 2018-01-16 Ricoh, Co., Ltd. Analyzing usage of visual content to determine relationships indicating unsuccessful attempts to retrieve the visual content
US9418122B2 (en) 2006-08-25 2016-08-16 Surf Canyon Incorporated Adaptive user interface for real-time search relevance feedback
US20140032311A1 (en) * 2006-08-25 2014-01-30 Surf Canyon Incorporated Adaptive user interface for real-time search relevance feedback
US8924378B2 (en) * 2006-08-25 2014-12-30 Surf Canyon Incorporated Adaptive user interface for real-time search relevance feedback
US7577643B2 (en) * 2006-09-29 2009-08-18 Microsoft Corporation Key phrase extraction from query logs
US20080082477A1 (en) * 2006-09-29 2008-04-03 Microsoft Corporation Key phrase extraction from query logs
US9037581B1 (en) * 2006-09-29 2015-05-19 Google Inc. Personalized search result ranking
US8762373B1 (en) 2006-09-29 2014-06-24 Google Inc. Personalized search result ranking
US20130124512A1 (en) * 2006-10-11 2013-05-16 Collarity, Inc. Negative associations for generation of refinement options
US9235627B1 (en) 2006-11-02 2016-01-12 Google Inc. Modifying search result ranking based on implicit user feedback
US11188544B1 (en) 2006-11-02 2021-11-30 Google Llc Modifying search result ranking based on implicit user feedback
US11816114B1 (en) 2006-11-02 2023-11-14 Google Llc Modifying search result ranking based on implicit user feedback
US10229166B1 (en) 2006-11-02 2019-03-12 Google Llc Modifying search result ranking based on implicit user feedback
US9811566B1 (en) 2006-11-02 2017-11-07 Google Inc. Modifying search result ranking based on implicit user feedback
US8661029B1 (en) 2006-11-02 2014-02-25 Google Inc. Modifying search result ranking based on implicit user feedback
US9110975B1 (en) * 2006-11-02 2015-08-18 Google Inc. Search result inputs using variant generalized queries
US20130254031A1 (en) * 2006-12-12 2013-09-26 International Business Machines Corporation Dynamic Modification of Advertisements Displayed in Response to a Search Engine Query
US8484199B1 (en) * 2006-12-12 2013-07-09 Google Inc. Ranking of geographic information
US9690805B1 (en) 2006-12-12 2017-06-27 Google Inc. Ranking of geographic information
US10360228B1 (en) 2006-12-12 2019-07-23 Google Llc Ranking of geographic information
US9684727B1 (en) 2006-12-12 2017-06-20 Google Inc. Ranking of geographic information
US8745041B1 (en) * 2006-12-12 2014-06-03 Google Inc. Ranking of geographic information
US8645817B1 (en) 2006-12-29 2014-02-04 Monster Worldwide, Inc. Apparatuses, methods and systems for enhanced posted listing generation and distribution management
US20140188743A1 (en) * 2006-12-29 2014-07-03 Monster Worldwide, Inc. Apparatuses, methods and systems for enhanced posted listing generation and distribution management
US20080168045A1 (en) * 2007-01-10 2008-07-10 Microsoft Corporation Content rank
US8938463B1 (en) 2007-03-12 2015-01-20 Google Inc. Modifying search result ranking based on implicit user feedback and a model of presentation bias
US9116963B2 (en) 2007-03-13 2015-08-25 Google Inc. Systems and methods for promoting personalized search results based on personal information
US7827170B1 (en) 2007-03-13 2010-11-02 Google Inc. Systems and methods for demoting personalized search results based on personal information
US8620915B1 (en) * 2007-03-13 2013-12-31 Google Inc. Systems and methods for promoting personalized search results based on personal information
US8694374B1 (en) 2007-03-14 2014-04-08 Google Inc. Detecting click spam
US8612460B1 (en) * 2007-03-27 2013-12-17 Google Inc. Content entity management
US20080244428A1 (en) * 2007-03-30 2008-10-02 Yahoo! Inc. Visually Emphasizing Query Results Based on Relevance Feedback
US20080263025A1 (en) * 2007-04-20 2008-10-23 Koran Joshua M Use of natural search click events to augment online campaigns
US7844605B2 (en) * 2007-04-20 2010-11-30 Yahoo! Inc. Using natural search click events to optimize online advertising campaigns
US20080270473A1 (en) * 2007-04-30 2008-10-30 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Determining an influence on a person by web pages
US9092510B1 (en) 2007-04-30 2015-07-28 Google Inc. Modifying search result ranking based on a temporal element of user feedback
US20130254802A1 (en) * 2007-06-27 2013-09-26 Google Inc. Selection of advertisements for placement with content
US10192279B1 (en) 2007-07-11 2019-01-29 Ricoh Co., Ltd. Indexed document modification sharing with mixed media reality
US8694511B1 (en) 2007-08-20 2014-04-08 Google Inc. Modifying search result ranking based on populations
US20090089311A1 (en) * 2007-09-28 2009-04-02 Yahoo! Inc. System and method for inclusion of history in a search results page
US8909655B1 (en) 2007-10-11 2014-12-09 Google Inc. Time based ranking
US9152678B1 (en) 2007-10-11 2015-10-06 Google Inc. Time based ranking
US9348912B2 (en) 2007-10-18 2016-05-24 Microsoft Technology Licensing, Llc Document length as a static relevance feature for ranking search results
US20090119254A1 (en) * 2007-11-07 2009-05-07 Cross Tiffany B Storing Accessible Histories of Search Results Reordered to Reflect User Interest in the Search Results
US20090119278A1 (en) * 2007-11-07 2009-05-07 Cross Tiffany B Continual Reorganization of Ordered Search Results Based on Current User Interaction
US20090158146A1 (en) * 2007-12-13 2009-06-18 Concert Technology Corporation Resizing tag representations or tag group representations to control relative importance
US20090319514A1 (en) * 2008-03-26 2009-12-24 Thales Method and system for assigning scores
US8090696B2 (en) * 2008-03-26 2012-01-03 Thales Method and system for assigning scores to elements in a set of structured data
US20120072457A1 (en) * 2008-04-10 2012-03-22 Abo Enterprises, Llc Fuzzy keyword searching
US20090259647A1 (en) * 2008-04-10 2009-10-15 Concert Technology Corporation Fuzzy keyword searching
US7958107B2 (en) * 2008-04-10 2011-06-07 Abo Enterprises, Llc Fuzzy keyword searching
US8375053B2 (en) * 2008-04-10 2013-02-12 Abo Enterprises, Llc Fuzzy keyword searching
US20140229812A1 (en) * 2008-04-11 2014-08-14 Advertising.Com Llc Systems and methods for video content association
US10970467B2 (en) 2008-04-11 2021-04-06 Verizon Media Inc. Systems and methods for video content association
US8812493B2 (en) 2008-04-11 2014-08-19 Microsoft Corporation Search results ranking using editing distance and document information
US10387544B2 (en) * 2008-04-11 2019-08-20 Oath (Americas) Inc. Systems and methods for video content association
US10387837B1 (en) 2008-04-21 2019-08-20 Monster Worldwide, Inc. Apparatuses, methods and systems for career path advancement structuring
US9830575B1 (en) 2008-04-21 2017-11-28 Monster Worldwide, Inc. Apparatuses, methods and systems for advancement path taxonomy
US9779390B1 (en) 2008-04-21 2017-10-03 Monster Worldwide, Inc. Apparatuses, methods and systems for advancement path benchmarking
US20090282021A1 (en) * 2008-05-12 2009-11-12 Bennett James D Web browser accessible search engine which adapts based on user interaction
US8886637B2 (en) * 2008-05-12 2014-11-11 Enpulz, L.L.C. Web browser accessible search engine which adapts based on user interaction
US20090292685A1 (en) * 2008-05-22 2009-11-26 Microsoft Corporation Video search re-ranking via multi-graph propagation
US8543570B1 (en) * 2008-06-10 2013-09-24 Surf Canyon Incorporated Adaptive user interface for real-time search relevance feedback
US20100017392A1 (en) * 2008-07-18 2010-01-21 Jianwei Dian Intent match search engine
US20100057714A1 (en) * 2008-09-04 2010-03-04 John Edward Miedema Search results ranking method and system
US8250066B2 (en) * 2008-09-04 2012-08-21 International Business Machines Corporation Search results ranking method and system
KR101683311B1 (en) * 2008-09-10 2016-12-06 마이크로소프트 테크놀로지 라이센싱, 엘엘씨 A method, system and computer-readable storage medium for providing information by using document length as a static relevance feature for ranking search results
KR20120089560A (en) * 2008-09-10 2012-08-13 마이크로소프트 코포레이션 Document length as a static relevance feature for ranking search results
US20100082610A1 (en) * 2008-09-30 2010-04-01 Peter Anick Query refinement based on user selections
US8819003B2 (en) 2008-09-30 2014-08-26 Yahoo! Inc. Query refinement based on user selections
US8452793B2 (en) * 2008-09-30 2013-05-28 Yahoo! Inc. Query refinement based on user selections
US20100082566A1 (en) * 2008-10-01 2010-04-01 Microsoft Corporation Evaluating the ranking quality of a ranked list
US9449078B2 (en) * 2008-10-01 2016-09-20 Microsoft Technology Licensing, Llc Evaluating the ranking quality of a ranked list
US20100131835A1 (en) * 2008-11-22 2010-05-27 Srihari Kumar System and methods for inferring intent of website visitors and generating and packaging visitor information for distribution as sales leads or market intelligence
US10380634B2 (en) * 2008-11-22 2019-08-13 Callidus Software, Inc. Intent inference of website visitors and sales leads package generation
US8396865B1 (en) 2008-12-10 2013-03-12 Google Inc. Sharing search engine relevance data between corpora
US8898152B1 (en) 2008-12-10 2014-11-25 Google Inc. Sharing search engine relevance data
US10158823B2 (en) * 2008-12-31 2018-12-18 Tivo Solutions Inc. Methods and techniques for adaptive search
JP2012514793A (en) * 2008-12-31 2012-06-28 ティヴォ インク Methods and techniques for adaptive search
US9152300B2 (en) 2008-12-31 2015-10-06 Tivo Inc. Methods and techniques for adaptive search
US10754892B2 (en) 2008-12-31 2020-08-25 Tivo Solutions Inc. Methods and techniques for adaptive search
US9037999B2 (en) 2008-12-31 2015-05-19 Tivo Inc. Adaptive search result user interface
US20110179453A1 (en) * 2008-12-31 2011-07-21 Poniatowski Robert F Methods and techniques for adaptive search
US20100198822A1 (en) * 2008-12-31 2010-08-05 Shelly Glennon Methods and techniques for adaptive search
US20100199219A1 (en) * 2008-12-31 2010-08-05 Robert Poniatowski Adaptive search result user interface
US20100208984A1 (en) * 2009-02-13 2010-08-19 Microsoft Corporation Evaluating related phrases
US20100241624A1 (en) * 2009-03-20 2010-09-23 Microsoft Corporation Presenting search results ordered using user preferences
US8577875B2 (en) 2009-03-20 2013-11-05 Microsoft Corporation Presenting search results ordered using user preferences
US9009146B1 (en) 2009-04-08 2015-04-14 Google Inc. Ranking search results based on similar queries
US8972394B1 (en) 2009-07-20 2015-03-03 Google Inc. Generating a related set of documents for an initial set of documents
US8977612B1 (en) 2009-07-20 2015-03-10 Google Inc. Generating a related set of documents for an initial set of documents
US8713003B2 (en) 2009-07-24 2014-04-29 Peer Belt Inc. System and method for ranking content and applications through human assistance
WO2011011147A3 (en) * 2009-07-24 2011-04-28 Krassimir Fotev System and method for ranking documents through human assistance
US20110022578A1 (en) * 2009-07-24 2011-01-27 Krassimir Fotev System and method for ranking documents through human assistance
EP2457179A4 (en) * 2009-07-24 2015-11-18 Krassimir Fotev System and method for ranking documents through human assistance
US8352464B2 (en) 2009-07-24 2013-01-08 Peer Belt Inc. System and method for ranking documents through human assistance
US20140040754A1 (en) * 2009-08-27 2014-02-06 Apple Inc. Adaptive mapping of search results
US9983765B2 (en) * 2009-08-27 2018-05-29 Apple Inc. Adaptive mapping of search results
US9697259B1 (en) 2009-08-31 2017-07-04 Google Inc. Refining search results
US9418104B1 (en) 2009-08-31 2016-08-16 Google Inc. Refining search results
US8738596B1 (en) 2009-08-31 2014-05-27 Google Inc. Refining search results
US8498974B1 (en) 2009-08-31 2013-07-30 Google Inc. Refining search results
US20130138637A1 (en) * 2009-09-21 2013-05-30 Walter Bachtiger Systems and methods for ranking media files
US8972391B1 (en) 2009-10-02 2015-03-03 Google Inc. Recent interest based relevance scoring
US9390143B2 (en) 2009-10-02 2016-07-12 Google Inc. Recent interest based relevance scoring
US8898153B1 (en) 2009-11-20 2014-11-25 Google Inc. Modifying scoring data based on historical changes
US9781202B2 (en) 2010-01-19 2017-10-03 Collarity, Inc. Anchoring for content synchronization
US8615514B1 (en) 2010-02-03 2013-12-24 Google Inc. Evaluating website properties by partitioning user feedback
US8924379B1 (en) 2010-03-05 2014-12-30 Google Inc. Temporal-based score adjustments
US8959093B1 (en) 2010-03-15 2015-02-17 Google Inc. Ranking search results based on anchors
US8639567B2 (en) 2010-03-19 2014-01-28 Visa U.S.A. Inc. Systems and methods to identify differences in spending patterns
WO2011116296A2 (en) * 2010-03-19 2011-09-22 Visa U.S.A. Inc. Systems and methods to enhance search data with transaction based data
WO2011116296A3 (en) * 2010-03-19 2012-01-19 Visa U.S.A. Inc. Systems and methods to enhance search data with transaction based data
US9799078B2 (en) 2010-03-19 2017-10-24 Visa U.S.A. Inc. Systems and methods to enhance search data with transaction based data
WO2011116301A2 (en) * 2010-03-19 2011-09-22 Visa U.S.A. Inc. Systems and methods to distribute advertisement opportunities to merchants
US20110231396A1 (en) * 2010-03-19 2011-09-22 Avaya Inc. System and method for providing predictive contacts
WO2011116301A3 (en) * 2010-03-19 2012-01-19 Visa U.S.A. Inc. Systems and methods to distribute advertisement opportunities to merchants
US8738418B2 (en) 2010-03-19 2014-05-27 Visa U.S.A. Inc. Systems and methods to enhance search data with transaction based data
US11017482B2 (en) 2010-03-19 2021-05-25 Visa U.S.A. Inc. Systems and methods to enhance search data with transaction based data
US9953373B2 (en) 2010-03-19 2018-04-24 Visa U.S.A. Inc. Systems and methods to enhance search data with transaction based data
US10360605B2 (en) * 2010-03-29 2019-07-23 Rakuten, Inc. Server apparatus, information providing method, information providing program, recording medium recording the information providing program, and information providing system
US8861844B2 (en) 2010-03-29 2014-10-14 Ebay Inc. Pre-computing digests for image similarity searching of image-based listings in a network-based publication system
US11605116B2 (en) 2010-03-29 2023-03-14 Ebay Inc. Methods and systems for reducing item selection error in an e-commerce environment
US9280563B2 (en) 2010-03-29 2016-03-08 Ebay Inc. Pre-computing digests for image similarity searching of image-based listings in a network-based publication system
US20110235902A1 (en) * 2010-03-29 2011-09-29 Ebay Inc. Pre-computing digests for image similarity searching of image-based listings in a network-based publication system
US9471604B2 (en) 2010-03-29 2016-10-18 Ebay Inc. Finding products that are similar to a product selected from a plurality of products
US8949252B2 (en) 2010-03-29 2015-02-03 Ebay Inc. Product category optimization for image similarity searching of image-based listings in a network-based publication system
US11132391B2 (en) 2010-03-29 2021-09-28 Ebay Inc. Finding products that are similar to a product selected from a plurality of products
US9529919B2 (en) 2010-03-29 2016-12-27 Paypal, Inc. Traffic driver for suggesting stores
US10528615B2 (en) 2010-03-29 2020-01-07 Ebay, Inc. Finding products that are similar to a product selected from a plurality of products
US9405773B2 (en) * 2010-03-29 2016-08-02 Ebay Inc. Searching for more products like a specified product
US20110238659A1 (en) * 2010-03-29 2011-09-29 Ebay Inc. Two-pass searching for image similarity of digests of image-based listings in a network-based publication system
EP2558994A4 (en) * 2010-04-14 2014-01-22 Microsoft Corp Search advertisement selection based on user actions
EP2558994A2 (en) * 2010-04-14 2013-02-20 Microsoft Corporation Search advertisement selection based on user actions
WO2011130019A2 (en) 2010-04-14 2011-10-20 Microsoft Corporation Search advertisement selection based on user actions
CN102822858A (en) * 2010-04-14 2012-12-12 微软公司 Search advertisement selection based on user actions
US8738635B2 (en) 2010-06-01 2014-05-27 Microsoft Corporation Detection of junk in search result ranking
US9623119B1 (en) 2010-06-29 2017-04-18 Google Inc. Accentuating search results
US8832083B1 (en) 2010-07-23 2014-09-09 Google Inc. Combining user feedback
US11295374B2 (en) 2010-08-28 2022-04-05 Ebay Inc. Multilevel silhouettes in an online shopping environment
US20120143790A1 (en) * 2010-12-01 2012-06-07 Microsoft Corporation Relevance of search results determined from user clicks and post-click user behavior obtained from click logs
US9002867B1 (en) 2010-12-30 2015-04-07 Google Inc. Modifying ranking data based on document changes
US8732151B2 (en) 2011-04-01 2014-05-20 Microsoft Corporation Enhanced query rewriting through statistical machine translation
US9762528B2 (en) 2011-07-27 2017-09-12 Ricoh Co., Ltd. Generating a conversation in a social network based on mixed media object context
US20130086083A1 (en) * 2011-09-30 2013-04-04 Microsoft Corporation Transferring ranking signals from equivalent pages
USD703687S1 (en) 2011-12-28 2014-04-29 Target Brands, Inc. Display screen with graphical user interface
USD711399S1 (en) 2011-12-28 2014-08-19 Target Brands, Inc. Display screen with graphical user interface
USD706794S1 (en) 2011-12-28 2014-06-10 Target Brands, Inc. Display screen with graphical user interface
USD706793S1 (en) 2011-12-28 2014-06-10 Target Brands, Inc. Display screen with graphical user interface
USD715818S1 (en) 2011-12-28 2014-10-21 Target Brands, Inc. Display screen with graphical user interface
USD711400S1 (en) 2011-12-28 2014-08-19 Target Brands, Inc. Display screen with graphical user interface
USD705792S1 (en) * 2011-12-28 2014-05-27 Target Brands, Inc. Display screen with graphical user interface
USD703685S1 (en) 2011-12-28 2014-04-29 Target Brands, Inc. Display screen with graphical user interface
USD703686S1 (en) 2011-12-28 2014-04-29 Target Brands, Inc. Display screen with graphical user interface
USD705790S1 (en) 2011-12-28 2014-05-27 Target Brands, Inc. Display screen with graphical user interface
US9495462B2 (en) 2012-01-27 2016-11-15 Microsoft Technology Licensing, Llc Re-ranking search results
US8954463B2 (en) * 2012-02-29 2015-02-10 International Business Machines Corporation Use of statistical language modeling for generating exploratory search results
US8954466B2 (en) * 2012-02-29 2015-02-10 International Business Machines Corporation Use of statistical language modeling for generating exploratory search results
US10789417B1 (en) * 2012-05-24 2020-09-29 The Boeing Company Method and apparatus for identifying relationships between documents
US20150112984A1 (en) * 2012-07-13 2015-04-23 Sony Computer Entertainment Inc. Processing apparatus
US9805096B2 (en) * 2012-07-13 2017-10-31 Sony Interactive Entertainment Inc. Processing apparatus
US20140122475A1 (en) * 2012-10-29 2014-05-01 Alibaba Group Holding Limited Search result ranking method and system
US20170255630A1 (en) * 2012-10-29 2017-09-07 Alibaba Group Holding Limited Search result ranking method and system
US9633082B2 (en) * 2012-10-29 2017-04-25 Alibaba Group Holding Limited Search result ranking method and system
TWI567673B (en) * 2012-10-29 2017-01-21 Alibaba Group Services Ltd Sorting methods and devices for search results
CN103793388A (en) * 2012-10-29 2014-05-14 阿里巴巴集团控股有限公司 Method and device for search result sorting
CN103164521A (en) * 2013-03-11 2013-06-19 亿赞普(北京)科技有限公司 Keyword calculation method and device based on user browse and search actions
US9183499B1 (en) 2013-04-19 2015-11-10 Google Inc. Evaluating quality based on neighbor features
US11157576B2 (en) * 2013-07-31 2021-10-26 Tencent Technology (Shenzhen) Company Limited Method, system and terminal for performing search in a browser
US9348945B2 (en) * 2013-08-29 2016-05-24 Google Inc. Modifying search results based on dismissal action associated with one or more of the search results
US20150066973A1 (en) * 2013-08-29 2015-03-05 Google Inc. Modifying Search Results Based on Dismissal Action Associated With One or More of the Search Results
US10896461B2 (en) * 2014-06-25 2021-01-19 Beijing Baidu Netcom Science And Technology Co., Ltd. Method and apparatus for data mining based on users' search behavior
CN105069102A (en) * 2015-08-03 2015-11-18 百度在线网络技术(北京)有限公司 Information push method and apparatus
WO2017020451A1 (en) * 2015-08-03 2017-02-09 百度在线网络技术(北京)有限公司 Information push method and device
US11216503B1 (en) 2015-09-08 2022-01-04 Google Llc Clustering search results
US10496691B1 (en) 2015-09-08 2019-12-03 Google Llc Clustering search results
US10282358B2 (en) 2015-09-30 2019-05-07 Yandex Europe Ag Methods of furnishing search results to a plurality of client devices via a search engine system
US10956834B2 (en) 2016-02-23 2021-03-23 Splunk Inc. Tool for machine-learning data analysis
US10607150B2 (en) 2016-02-23 2020-03-31 Splunk Inc. Machine-learning data analysis tool
US10679142B2 (en) * 2016-04-29 2020-06-09 Splunk Inc. Dynamic search guidance for machine data indexing and search system
US10387559B1 (en) * 2016-11-22 2019-08-20 Google Llc Template-based identification of user interest
CN108108380A (en) * 2016-11-25 2018-06-01 阿里巴巴集团控股有限公司 Search ordering method, searching order device, searching method and searcher
US10681029B2 (en) * 2016-11-28 2020-06-09 Yandex Europe Ag Method of selecting a website for displaying in a web browser quick-access field
US20180152431A1 (en) * 2016-11-28 2018-05-31 Yandex Europe Ag Method of selecting a website for displaying in a web browser quick-access field
WO2019007985A1 (en) * 2017-07-05 2019-01-10 Technische Universität Darmstadt Method and device for flexible ranking system for information in a multi-linked network
US20220414167A1 (en) * 2019-04-30 2022-12-29 Capital One Services, Llc Techniques to leverage machine learning for search engine optimization
US11887184B2 (en) 2019-05-02 2024-01-30 Capital One Services, Llc Techniques to facilitate online commerce by leveraging user activity
CN110781383A (en) * 2019-09-27 2020-02-11 上海麦克风文化传媒有限公司 Method for determining comprehensive sequencing investigation index
CN112597361A (en) * 2020-12-16 2021-04-02 北京五八信息技术有限公司 Sorting processing method and device, electronic equipment and storage medium
US11935103B2 (en) 2021-12-29 2024-03-19 Ebay Inc. Methods and systems for reducing item selection error in an e-commerce environment

Also Published As

Publication number Publication date
US20100106703A1 (en) 2010-04-29
WO2007130914A2 (en) 2007-11-15
US20120078710A1 (en) 2012-03-29
US8095582B2 (en) 2012-01-10
WO2007130914A3 (en) 2008-08-28

Similar Documents

Publication Publication Date Title
US8095582B2 (en) Dynamic search engine results employing user behavior
US8442973B2 (en) Real time implicit user modeling for personalized search
US20160350428A1 (en) Real time implicit user modeling for personalized search
US11036814B2 (en) Search engine that applies feedback from users to improve search results
US10387462B2 (en) Feature engineering and user behavior analysis
US9418122B2 (en) Adaptive user interface for real-time search relevance feedback
US7693827B2 (en) Personalization of placed content ordering in search results
US11681750B2 (en) System and method for providing content to users based on interactions by similar other users
US8060456B2 (en) Training a search result ranker with automatically-generated samples
US8515950B2 (en) Combining log-based rankers and document-based rankers for searching
US8321278B2 (en) Targeted advertisements based on user profiles and page profile
US7996400B2 (en) Identification and use of web searcher expertise
CN105912669B (en) Method and device for complementing search terms and establishing individual interest model
US20080120289A1 (en) Method and systems for real-time active refinement of search results
CA2571867A1 (en) Results based personalization of advertisements in a search engine
US20070276812A1 (en) Search Result Ranking Based on Usage of Search Listing Collections
WO2011018453A1 (en) Method and apparatus for searching documents

Legal Events

Date Code Title Description
AS Assignment

Owner name: SURF CANYON, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CRAMER, MARK;REEL/FRAME:018510/0420

Effective date: 20061025

STCB Information on status: application discontinuation

Free format text: EXPRESSLY ABANDONED -- DURING EXAMINATION