WO2007008809A2 - Visually-oriented driving directions in digital mapping system - Google Patents
Visually-oriented driving directions in digital mapping system Download PDFInfo
- Publication number
- WO2007008809A2 WO2007008809A2 PCT/US2006/026745 US2006026745W WO2007008809A2 WO 2007008809 A2 WO2007008809 A2 WO 2007008809A2 US 2006026745 W US2006026745 W US 2006026745W WO 2007008809 A2 WO2007008809 A2 WO 2007008809A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- waypoints
- driving directions
- images
- scored
- waypoint
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/36—Input/output arrangements for on-board computers
- G01C21/3602—Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/36—Input/output arrangements for on-board computers
- G01C21/3626—Details of the output of route guidance instructions
- G01C21/3644—Landmark guidance, e.g. using POIs or conspicuous other objects
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/36—Input/output arrangements for on-board computers
- G01C21/3626—Details of the output of route guidance instructions
- G01C21/3647—Guidance involving output of stored or live camera images or video streams
Definitions
- the invention relates to digital mapping systems, and more particularly, to computer generated driving directions that are visually-oriented.
- Computerized mapping systems have been developed to search for, identify, and discover information about geographic locations.
- One form of such computerized mapping systems includes travel-planning Internet websites. With an excess of 50 million unique monthly users, such map sites are a very popular offering. Examples of such sites include AOL's MapQuest, Yahoo's Telcontar-based maps, and Microsoft's MapPoint.net suite. Such sites all work along the lines of a common model, as will now be described.
- a Web user asks for a new map view (e.g., by entering a postal address, or by clicking a navigation link next to a current map view), the user's Web browser sends to a Web server a request indicating the boundaries of the new map view.
- the Web server in turn extracts the corresponding vector-based map data from a database, and draws a bitmap image of the map.
- the server then converts the bitmap to an image format supported by the user's Web browser and returns the image, sometimes embedded in HTML, to the user's Web browser so that it can be displayed.
- Other map Web sites such as Britain's MultiMaps or Australia's Wherels utilize a raster-based map database instead. In these cases, it is not necessary to extract vectors and draw a map image. Rather, these functions are replaced by simply extracting the appropriate part of a larger, pre-rendered image.
- One embodiment of the present invention provides a computer implemented method for generating visually-oriented driving directions.
- the method includes constructing a mapping between business listings and at least one of satellite images and storefront images of a target map area.
- the method continues with analyzing the at least one of satellite images and storefront images to identify waypoints, and evaluating distinctiveness of the identified waypoints, thereby providing a number of scored waypoints.
- the method further includes incorporating one or more of the scored waypoints into requested driving directions.
- the method may include applying OCR to storefront images to identify waypoints (e.g., to identify text and signage).
- the method may include performing visual recognition of chain stores using the at least one of satellite images and storefront images to identify waypoints (e.g., to recognize patterns of branding and architecture).
- the method may include performing non- visual recognition of chain stores using at least one of heuristics and business listings to identify waypoints (e.g., to recognize well-known and visually predictable businesses).
- the method may include storing the scored waypoints.
- the constructing, analyzing, evaluating, and storing are performed in advance of receiving requests for driving directions.
- Evaluating distinctiveness of the identified waypoints may include, for example, assigning a distinctiveness score based on at least one of a difference between a waypoint and its surroundings, scope of the difference, salience of the waypoint, visibility of the waypoint, and familiarity of pattern associated with the waypoint.
- the method may include adjusting a distinctiveness score of one or more of the scored waypoints based on user feedback.
- Incorporating one or more of the scored waypoints into requested driving directions may further include serving those driving directions to the requestor.
- the method may include providing a visual preview of the driving directions using at least one of satellite images, storefront images, heuristics, and business listings.
- Incorporating one or more of the scored waypoints into requested driving directions may include selecting waypoints related to a destination of the requestor.
- One or more of the scored waypoints can be associated with a business advertising arrangement. In one such configuration, a cost-per-use of the one or more scored waypoints associated with a business advertisement is a factor that impacts waypoint distinctiveness scoring.
- Another embodiment of the present invention provides a computer implemented method for generating visually-oriented driving directions.
- the method includes constructing a mapping between business listings and at least one of satellite images and storefront images of a target map area, analyzing the at least one of satellite images and storefront images to identify waypoints, and evaluating distinctiveness of the identified waypoints, thereby providing a number of scored waypoints.
- the method further includes storing the scored waypoints for subsequent use in computer-generated driving directions.
- the method may include at least one of applying OCR to storefront images to identify waypoints, performing visual recognition of chain stores using the at least one of satellite images and storefront images to identify waypoints, and performing non- visual recognition of chain stores using at least one of heuristics and business listings to identify waypoints.
- Evaluating distinctiveness of the identified waypoints may include, for example, assigning a distinctiveness score based on at least one of a difference between a waypoint and its surroundings, scope of the difference, salience of the waypoint, visibility of the waypoint, and familiarity of pattern associated with the waypoint.
- the method may include adjusting a distinctiveness score of one or more of the scored waypoints based on user feedback.
- the method may include incorporating one or more of the scored waypoints into requested driving directions, providing a preview of the requested driving directions using the at least one of satellite images and storefront images, and serving the driving directions to the requestor.
- the method may include selecting one or more waypoints related to a destination of a user requesting driving directions, and incorporating the selected scored waypoints into the requested driving directions.
- One or more of the scored waypoints can be associated with a business advertising arrangement. In one such case, a cost-per-use of the one or more of the waypoints associated with a business advertisement is a factor that impacts way
- Another embodiment of the present invention provides a computer implemented method for generating visually-oriented driving directions.
- the method includes identifying distinctive waypoints associated with an area covered by a digital mapping system, wherein the waypoints are visual data points along one or more driving routes and are in addition to road names (e.g., Main Street, RoutelOla, and Highway 1) and road topology (e.g., bridges, traffic circles, on-ramp/off-ramp, paved/unpaved, tolls).
- the method further includes incorporating one or more of the distinctive waypoints into computer-generated driving directions produced by the digital mapping system in response to a user request, thereby providing visually-oriented driving directions.
- Each of these methods can be implemented, for example, using a tile-based mapping system.
- other systems such as non-tile vector-based and raster-based mapping systems can be used to implement the methods as well, as will be apparent in light of this disclosure.
- Figure 1 is a block diagram of a digital mapping system configured to provide visually-oriented driving directions, in accordance with one embodiment of the present invention.
- Figure 2 is a block diagram of the location data serving system shown in Figure 1, configured for generating visually-oriented driving directions in accordance with one embodiment of the present invention.
- Figure 3 is a block diagram of the waypoint identification and scoring module shown in Figure 2, configured for generating scored waypoints for use in visually-oriented driving directions, in accordance with one embodiment of the present invention.
- Figure 4 illustrates a method for generating visually-oriented driving directions, in accordance with one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION
- Digital mapping techniques are disclosed that provide visually-oriented information to the user, such as driving directions, thereby improving the user experience.
- mapping system configured in accordance with an embodiment of the presenting invention can give more visually-oriented (and hence human-friendly) directions.
- satellite-based imagery such as that provided by Google Earth or Digital Globe
- way points e.g., a particularly large building, an isolated building, a baseball or other large stadium, a park in an otherwise urban area.
- structures such as traffic lights and stop signs can be identified, which are also useful in giving directions.
- storefront (street-level) images can be used to identify salient buildings and features, such as stores with large visible logos, stores of unusual colors (e.g., "the bright purple store”), stores that are easily recognized because their brands are well-known and/or their store architectures and tradedress are distinctive (e.g., "the McDonalds”).
- This last category involving well-known brands, trademarks, tradedress, etc. can be identifiable even without image information (e.g., most everyone knows what a McDonalds looks like).
- Such visual data can be used to identify the target address, such as "Fred's Shoe Repair is in the middle of the block, just past the bright purple store”.
- the user can preview the driving route by a simulated drive-through or "fly- through” using the satellite/street-level images, in conjunction with the relevant digital maps.
- the visual cues in the directions will remind the user of what to look for, and give that user a greater sense of confidence that he or she is on the right track. Numerous other benefits will be apparent in light of this disclosure.
- Figure 1 is a block diagram of a tile-based digital mapping system configured in accordance with one embodiment of the present invention.
- the system includes a computing device 103 connected to a network 105.
- Various serving systems are also connected to the network 105, including a web serving system 110, a tile serving system 115, and a location data serving system 120.
- Other computing devices 103 may also be in communication with the serving systems 110, 115, and 120 via the network 105.
- the computing device 103 may be any type of device configured for computing, such as a personal computer or laptop, a mobile phone, a personal digital assistant, a navigation system located in a vehicle, a handheld GPS system, and so on.
- the computing device 103 includes a browser (or other similar application) that allows the user to interface and communicate with other devices and systems on the network 105.
- browsers include Microsoft's Internet Explorer browser, Netscape's Navigator browser, Mozilla's Firefox browser, PalmSource's Web Browser, or any other browsing or application software capable of communicating with network 105.
- the computing device 103 can be any device that allows a user to access the serving systems 110, 115, and 120 via the network 105.
- the web serving system 110 is the part of an overall system that delivers the initial HTML (hypertext markup language), images, scripting language (e.g., JavaScript, JScript, Visual Basic Script), XSLT (extensible stylesheet language transformation), and other static elements that are used by the browser on computing device 103.
- this serving system 110 may include one or more servers operating under a load balancing scheme, with each server (or a combination of servers) configured to respond to and interact with the computing device 103.
- the web serving system 110 can be implemented with conventional technology, as will be apparent in light of this disclosure.
- the tile serving system 115 is the part of the overall system responsible for delivering individual map tiles in response to requests from the computing device 103, with each tile being uniquely defined, for example, by x, y and z values that coordinates to an overall tile-based map. Other tile identification schemes can be used as well.
- the tile serving system 115 may include one or more servers operating under a load balancing scheme, with each server (or a combination of servers) configured to respond to and interact with the computing device 103. Example architecture and functionality of the tile serving system 115 is further discussed in the previously incorporated U.S. Application No. 11/088,542.
- the location data serving system 120 is the part of the overall system that delivers location data of various forms to the computing device 103.
- Its functions include, for example, finding the geographic location of a street address, generating and formatting visually-oriented driving directions, and searching for location-specific results to a query (e.g., as with the Google Local Search service). Other services may also be provided.
- searching for location-specific results to a query e.g., as with the Google Local Search service.
- Other services may also be provided.
- the user enters a search string, it is put into a request by the computing device 103, and sent to the location data serving system 120 via the network 105.
- the location data serving system 120 determines what the request is for (e.g., generate driving directions with visual "fly-through” of route, or generate visually-oriented driving directions, or generate both visually-oriented driving directions and visual "fly-through” of route), and responds with the appropriate data from various sub-systems, such as geo-coders, routing engines, and local search indexes or databases (e.g., including a scored waypoint database), in a format that computing device 103 can use to present the data to the user (e.g., via a browser).
- Example architecture and functionality of the location data serving system 120 will be discussed in turn with reference to Figures 2-4.
- the network 105 may be any type of communications network, such as a local area network (e.g., intranet), wide area network (e.g., internet), or some combination thereof.
- the network 105 may be a direct connection between the computing device 103 and the serving systems 110, 115, and 120.
- the computing device 103, network 105, and/or serving systems 110, 115, and 120 may be in communication via any type of wired or wireless connection, using a wide variety of communication protocols.
- FIG 2 is a block diagram of the location data serving system 120 shown in Figure 1, configured for generating visually-oriented driving directions in accordance with one embodiment of the present invention.
- the system 120 includes a waypoint identification and scoring module 205, a scored waypoint database 210, and a driving direction generator 215.
- the location data serving system 120 can be part of the overall digital mapping system that delivers mapping data of various forms (e.g., location data, map images, etc) to the computing device 103.
- the location data is in the form of visually-oriented driving directions. Note that these driving directions can be provided in conjunction with a map that includes an overlay of the driving directions in graphical form as well.
- the digital mapping system is configured to provide tile-based maps, as explained in detail in the previously incorporated U.S. Application Nos. 11/088,542 and 11/051,534.
- the present invention is not intended to be limited to systems that provided tile-based maps.
- embodiments of the present invention can also be used with other systems, such as non-tile vector-based and raster-based mapping systems.
- other embodiments of the present invention can be used to provide visually-oriented driving directions (with no map). Numerous mapping system configurations can be used. The description herein focuses on the visually-oriented driving directions component of the system. Other components and map system functionality not shown or otherwise discussed, whether conventional or custom, can be used as desired in conjunction with visually-oriented driving directions, as will be apparent in light of this disclosure.
- the location data serving system 120 operates in both an off-line capacity, as well as in an on-line capacity.
- "scored waypoints" are generated by the waypoint identification and scoring module 205 in an offline process, and stored in the scored waypoint database 210.
- This off-line process effectively stocks the database 210 with waypoints and their corresponding distinctiveness scores.
- requests e.g., HTTP
- the request can be initiated, for example, by a user engaging a web browser of a computing device 103 to access the system.
- a waypoint is a visual data point along the way of a driving route, such as a building, business, park, billboard, or some other visually noticeable structure or place. The distinctiveness of the waypoint is indicative of its visual prominence.
- the digital mapping system in which the location data serving system 120 operates may also generate and serve graphical map data relevant to the client request.
- the client receives the requested graphical map data, and requests any map tiles it doesn't already have displayed or cached (e.g., as explained in the previously incorporated U.S. Application No. 11/051,534.
- the client draws and displays the map, along with the visually-oriented driving directions.
- the client side can also be used to draw (e.g., overlay) the graphical driving directions, location markers, etc on the map image. Note here that the distinctive waypoints can also be overlaid on the map, along with the route itself.
- FIG 3 is a block diagram of the waypoint identification and scoring module 205 shown in Figure 2, configured for generating scored waypoints for use in visually-oriented driving directions, in accordance with one embodiment of the present invention.
- the module 205 generates scored waypoints, which can then be stored in scored waypoint database 210. These scored waypoints can then be accessed by a driving direction generator to create visually-oriented driving directions.
- the module 205 includes a storefront image database 305, a satellite image database 310, a yellow pages database 315, an image registration module 320, a satellite image processor 325, a storefront image process 330, an optical character recognition (OCR) module 335, a non-visual recognition module 340, a visual recognition module 345, and a waypoint distinctiveness evaluation module 350.
- OCR optical character recognition
- the waypoint identification and scoring module 205 employs one or more a databases of images (e.g., storefront image database 305 and satellite image database 310), together with a corresponding database of business listings (e.g., yellow pages database 315).
- Each database can be structured to facilitate efficient access of data, and include various types of information.
- each street-level image e.g., digital photograph taken using a GPS-enable camera
- each satellite image stored in the satellite image database 310 can be indexed by geocode, and associated with corresponding GPS coordinates.
- the yellow pages database 315 can be structured, for example, as conventionally done.
- the illustrated databases are integrated into a single database.
- other databases or information sets could be included, such as a conventional white pages database or other such listing service databases.
- the image databases may include multiple views and/or zoom levels of each targeted area. For instance, one storefront image can be taken from an angle as it would be seen coming down one direction of the street, while another storefront image of the same address could be taken from an angle as it would be seen coming down the other direction of the street. Thus, depending on the driving directions, either image could be used.
- the image registration module 320 is programmed or otherwise configured to construct a mapping between images and business listings. In one embodiment, this mapping is accomplished by a combination of image segmentation using standard image-processing techniques (e.g., edge detection, etc.) and interpolation of a business's street address within the range of street addresses known to be contained in the image. Image registration is done for both storefront images and satellite images.
- the mapping can be implemented, for example, with a pointer or address scheme that effectively connects images from an image database to listings in the yellow pages database.
- a single database can be built as the image registration process is carried out, where the records of the single database are indexed by geocode, and each record includes image data and related yellow page listing information.
- image processing is performed by accessing the images by way of the image registration module 320 (e.g., which can access the images stored in the respective databases 305 and 310 using a pointer or addressing scheme). Other embodiments can access the images directly from their respective databases. In any case, image processing is performed to propose candidate waypoints based on detected visual features.
- the satellite image processor 325 is programmed or otherwise configured to recognize navigational features included in the satellite images. Certain navigational features are particularly useful for drivers, because a driver has to attend to such features anyway, including the likes of traffic lights, stop signs, on/off-ramps, traffic circles, city blocks, tunnels, bridges, end of divided highway, and other driver-centric features.
- the satellite image processor 325 uses standard machine-learning techniques to learn to recognize these navigational features from the satellite images.
- humans annotate a set of satellite images with instances of the desired feature (e.g., all traffic circles), and to then use standard machine- learning algorithms (e.g., SVMs, boosting over decision stumps, decision lists, etc.) to learn image features that correctly predict the occurrence of that feature.
- standard machine- learning algorithms e.g., SVMs, boosting over decision stumps, decision lists, etc.
- the machine-learning algorithms can be used to recognize the learned or known navigational features in all the satellite images. These identified features can then be evaluated for use as waypoints.
- the storefront image processor 330 is programmed or otherwise configured to analyze storefront images. In one embodiment, this analysis is carried out at both a coarse level (e.g., width, height, color histograms) and a more refined level (e.g., segmentation into facade, doors, windows, roof; architectural elements such as pillars and balconies; decorative elements such as awnings, signage, neon lights, painted designs). Such analysis can carried out, for example, using standard image-processing techniques (e.g., computer vision). Standard feature extraction algorithms typically extract high level information from images, such as shapes, colors, etc. Pattern recognition algorithms can then be applied to classify the extracted information so as to "recognize" objects in the storefront images.
- a coarse level e.g., width, height, color histograms
- a more refined level e.g., segmentation into facade, doors, windows, roof; architectural elements such as pillars and balconies; decorative elements such as awnings, signage, neon lights, painted designs.
- Standard image-processing techniques
- GPS and 3-D accelerometer readings could also be collected for each photographic image taken.
- Certain navigational features are associated with a particular set of associated readings. For example: on-ramps would have strong acceleration, and off- ramps would have strong deceleration; and traffic lights would have deceleration to zero, followed at regular intervals by acceleration.
- the GPS could be used to record where these acceleration/deceleration patterns occurred.
- waypoints can be proposed based on observed locally distinctive or otherwise salient features, including any measurable dimension or highly noticeable quality (e.g., unusually wide/narrow/tall/short building, building with purple door, orange roof, green neon sign, etc). If a feature is "locally distinctive", it is unique within some vicinity, such as the only pink building on the block. The features discussed so far are intrinsic to the building, but extrinsic features can also be used (e.g., the building at the corner, the first building over the railroad tracks, etc).
- the OCR module 335 can be applied to the storefront images to read the storefront signage (if any), using standard OCR algorithms and techniques. This text recognition enables waypoint proposal of visual features such as "the bar with the neon Karaoke sign in the window.”
- the OCR accuracy of module 335 can be improved by identifying what kind of a store is in the storefront image, based on its corresponding category listing (e.g., bars and restaurants) in the yellow pages database 315. Recall that the image registration module 320 has already mapped the images to corresponding listings within the yellow pages database 315, thereby facilitating this context identification for the OCR process.
- text related to that yellow pages category can be obtained, for example, by visiting web sites of stores in that category, and adjusting the language model used for OCR module 335, accordingly.
- This supplemental information from yellow page listings and/or websites enables the OCR module 335 to be informed of the context in which it is operating.
- the visual learning module 345 receives the proposed waypoint information output by each of the image processors 325 and 330, and the OCR module 335.
- the visual learning module 345 is programmed or otherwise configured to perform visual learning based on established knowledge. For instance, in one particular embodiment, the visual learning module 345 recognizes chain stores. Chain stores (e.g., McDonalds) are a common element in human directions.
- the visual learning module 345 can learn which chain stores have distinctive logos/buildings by, for example, identifying common elements across storefront images of multiple branches using the results of the storefront image processor 325 and OCR module 335, combined with yellow page listings and standard translation/rotation/scaling transformations to align multiple images, and even simpler image-processing techniques such as orientation histograms.
- the visual learning module 345 would discover that McDonalds frequently have the same logo (e.g., the word "McDonalds" in a particular color and font), a glass storefront, Golden Arches, etc. In this sense, the visual learning module 345 learns a prototype of what the target chain store typically looks like. In addition, the visual learning module 345 identifies the extent to which each branch of the chain store at a given address matches the prototype. If a McDonalds branch at a given address sufficiently matches the common McDonalds' features found in the prototype, that McDonalds branch is a useful waypoint to use in driving directions, because it will be easily recognized by humans as a McDonalds.
- the visual learning module 345 would discover that McDonalds frequently have the same logo (e.g., the word "McDonalds" in a particular color and font), a glass storefront, Golden Arches, etc. In this sense, the visual learning module 345 learns a prototype of what the target chain store typically looks like. In addition, the visual learning module 345 identifie
- a given chain-store branch may be actually be a "bad" waypoint if, for instance, it doesn't match the prototype well. For example, a McDonalds in an upscale neighborhood may be required to conform to the prevailing architecture (e.g., no large Golden Arches allowed), thus reducing its recognizability as a McDonalds.
- a given chain-store branch may actually be a "bad” waypoint if, for instance, that particular branch is not clearly visible from the road (e.g., obscured by foliage or other buildings).
- the visual learning module 345 will avoid using such branches of the chain store as waypoints, thereby further refining results from the image processors 325 and 330, and the OCR module 335.
- McDonalds is well-known and fairly consistent in its appearance (even when restricted by local ordinance), and may not need to be processed by the visual recognition module (e.g., McDonalds image and address data could simply pass through the visual learning module 345 as a proposed waypoint for a given area).
- the non-visual recognition module 340 can be used to identify McDonalds and other well- known architectures, as will be apparent in light of this disclosure.
- a more regional restaurant chain that is less well-known, but fairly consistent in its appearance, could be identified as a waypoint of comparable efficacy to McDonalds by the visual learning module 345.
- the non-visual recognition module 340 can be used to recognize chain stores (and other such consistent structures and the like) without reference to any images. Rather, only address information (e.g., from the yellow pages database 315 in this embodiment) is needed. For instance, instead of learning from images of multiple branches of a chain store, heuristics about store recognizability can be used.
- address information e.g., from the yellow pages database 315 in this embodiment
- heuristics about store recognizability can be used.
- gas stations are often good, recognizable waypoints because they tend to display their logo prominently, and they have distinctive architecture (e.g., canopy over gas pumps and ample signage).
- branches with a multitude of branches (e.g., over 1000) in the yellow pages tend to have distinctive branding and widespread marketing, and are typically well-recognized.
- chain grocery stores often have large logos, large buildings, and large parking lots. In addition, they are typically well-known in the communities in which they exist.
- the non-visual recognition module 340 uses address listings to identify chain stores and other predictable/consistent architectures that are known to be distinctive, and proposes them as waypoints.
- the waypoint distinctiveness evaluation module 350 evaluates the distinctiveness of the proposed waypoints received from the non-visual recognition module 340 and the visual recognition module 345.
- the distinctiveness measure can range, for example, from 1 to 10, and can take into account a number of aspects, where a distinctiveness score of 1 is indicative of a low distinctiveness rating, and a distinctiveness score of 10 is indicative of a high distinctiveness rating, hi one particular embodiment, one or more of the following factors are considered in assigning a distinctiveness score: the magnitude of difference of the waypoint and its surroundings such as neighbors and open space (e.g., bright red is far away from white in color-space); the scope of the difference (e.g., the only bright red building on an entire block is more distinctive than a bright red building with another bright red building 2 doors down); the salience of a waypoint (e.g., a bright red door is less salient than a bright red entire facade); the visibility of the waypoint (e.g., a building separated from its neighbors by an empty lot may be
- waypoints and their distinctiveness scores can be used in several ways when generating driving directions (e.g., using the driving directions generator 215). For instance, turns can be identified using waypoints at or near an intersection (e.g., "you'll see a McDonalds on your right and a Mobil station on your left” or "turn right just past the Home Depot on your right"), hi one particular embodiment, the waypoints that are given are on the same side of the street as the turn (e.g., waypoints on the right-hand side for right turns), so drivers can focus on one side of the road at a time.
- waypoints at or near an intersection e.g., "you'll see a McDonalds on your right and a Mobil station on your left” or "turn right just past the Home Depot on your right”
- the waypoints that are given are on the same side of the street as the turn (e.g., waypoints on the right-hand side for right turns), so drivers can focus on one side of the road at a time.
- Confirmatory waypoints can be provided, such as "You'll pass a Safeway on your left, then a large park on your right.” Also, "gone too far” waypoints can be given, such as "If you see a Tower Records on your right, you've gone too far.” Waypoints can also be used as early warning indicators, to signal the driver to start watching for a turn, especially if the driver has been on the same road for awhile ("you'll be on Center St. for 20 miles; after you pass the Crab tree Shopping Center on your left, start watching for your left turn”).
- Waypoints can also be used to identify the target destination (e.g., "your destination is the three-story white house on the left” or “you'll see the Fry's on your right; look for the big red logo”).
- the entire route can be chosen with the quality of waypoints in mind. For instance, the user can be directed along a somewhat longer route if the longer path affords better waypoints. In one particular embodiment, the user can be offered a choice of maximizing "ease of following directions” versus "shortest distance” (or other such alternatives).
- the waypoint distinctiveness evaluation module 350 is configured to score the salience of streets (e.g., "large street with two lanes in each directions" or "a boulevard with grassy median").
- the driving directions can be generated using conventional technology or as described in the previously incorporated U.S. Application Nos. 11/088,5.42 and 11/051,534, with the addition of the incorporation of one or more of the scored waypoints.
- the waypoints are incorporated into a pre-selected set of driving directions, where conventional driving directions are generated, and then effectively annotated with waypoints.
- These waypoints can be added to the driving directions as text (as in the previously given examples) and/or as images (e.g., McDonalds icon).
- driving directions can also be generated for a plurality of routes, and taking the waypoint scores along each route into account when selecting the best route to give.
- a mechanism for collecting feedback from users as to the utility of different waypoints is provided.
- a GPS record of the user's actual route taken when attempting to follow the served driving directions e.g., from a handheld GPS receiver/recorder, or an on-board GPS navigation system, or a GPS-enabled cellphone.
- the reporting of GPS data can be transparent to the user, or enabled by the user.
- a user feedback module is downloaded into the user's browser, and is programmed to receive the GPS data and report it back to the serving system.
- An alternate feedback approach that can be used is to provide a form (e.g., online webform or email, and/or paper/fax submissions) for users to fill out and send back with positive/negative feedback about each step of the visually-oriented driving directions.
- the user feedback could be used, for example, by the waypoint distinctiveness evaluation module 350 to reduce the distinctiveness score of waypoint(s) that were used to indicate a turn that the user missed, or to increase the distinctiveness score of waypoint(s) that were used to indicate a turn that the user found.
- Feedback from multiple users could be categorized and tallied by waypoint, so that a meaningful sample of feedback data could be analyzed when upgrading/downgrading the distinctiveness score of a waypoint, thereby eliminating or otherwise reducing the impact of anomalies or individual biases regarding a particular waypoint. Numerous statistical techniques can be used to properly value the user feedback.
- businesses are allowed to bid or otherwise pay to be included as a waypoint.
- the "cost-per-use" of a waypoint could then be an additional factor that would be taken into account when scoring each waypoint.
- incorporating one or more of the scored waypoints into requested driving directions could include selecting waypoints related to a destination of the requestor. For instance, if the user is asking for driving directions to a national park, then waypoints such as camping equipment stores could be selected for integration into the served driving directions.
- This choice of waypoints has several benefits: (1) the waypoints might be of interest in their own right; (2) the waypoints are more likely to be familiar to the user and thus better waypoints; and (3) advertisers would be likely to pay more for such targeted waypoint usage.
- fly-throughs e.g., using satellite images
- walkthroughs e.g., using storefront images
- photos of the waypoints and intersections could be printed in advance by the user to aid the user in spotting them when driving the actual route.
- an "autoplay preview" feature is provided on the client side. When selected by the user, a request for preview data would be sent to the server side.
- the client could initially be given the entire route, which can be represented, for example, as a sequence of latitude/longitude pairs of line segments approximating the path.
- the client can then use simple linear interpolation to follow the route.
- the client could display a map (e.g., satellite map or bitmap map) centered initially at the starting point of the route, with a progress marker (e.g., a car icon), and would continuously scroll the display along the desired route. Movement of the progress marker could be carried out using script code (e.g., JavaScript) executing on the client.
- the client could pre-fetch map tiles (or other map image) in conjunction with moving the progress marker to create an uninterrupted viewing experience.
- An analogous pre-fetching scheme could be used for displaying storefront images along the route, which may be more meaningful to the user, given the ease of perceiving street-level images.
- a perspective transformation can be used to show the storefronts on both sides of the street as the user drives (or walks) along the route.
- Other elements can be added to the preview as desired, such as traffic lights, street signs, and even simulated other cars, etc., to enhance realism.
- the user is able to control the playback (e.g., via start/stop/pause/reverse control objects included in the user interface).
- the user is allowed to divert from the original route, so as to explore alternate routes and the surrounding area ("virtual reconnaissance").
- predictive caching of the relevant map images/storefront images can be used based on the user's current direction and previous use habits.
- a number of predictive caching and image delivery schemes can be used, as will be apparent in light of this disclosure.
- Figure 4 illustrates a method for generating visually-oriented driving directions, in accordance with one embodiment of the present invention.
- the method can be carried out, for example, using the systems of Figures 1-3.
- any mapping system can integrate the methodology.
- the method includes an off-line portion and an on-line portion.
- the off-line portion includes constructing 405 a mapping between business listings and satellite and storefront images of the target map area.
- the method continues with analyzing 410 the satellite images to identify waypoints, and analyzing 415 the storefront images to identify waypoints.
- the method further includes applying 420 OCR to storefront images to identify waypoints.
- image analysis can be used to detect decorative and structural features within the images, and OCR can be used to detect the storefront signage.
- the off-line portion of the method continues with performing 425 visual recognition of chain stores using images (e.g., storefront and satellite) to identify waypoints.
- the off-line portion of the method also includes performing 430 non- visual recognition of chain stores using heuristics and/or business listings to identify waypoints. As previously explained, there are many rules of thumb that can be applied to identify waypoints, without using image data. In addition, both visual (e.g., machine learning) and non-visual (e.g., GPS/acceleration data collection) techniques can be used to identify navigational features, which can also be used as waypoints.
- visual e.g., machine learning
- non-visual e.g., GPS/acceleration data collection
- the method continues with evaluating 435 the distinctiveness of the proposes waypoints.
- the distinctiveness measure can range, for example, from 1 (low distinctiveness) to 10 (high distinctiveness), and takes into account a number of factors as previously explained. Numerous rating schemes are possible, and the end result is a number of scored waypoints.
- the off-line portion of the method further includes storing 440 the scored waypoints
- the on-line portion of the method includes incorporating 445 one or more scored waypoints into requested driving directions.
- a driving direction generator on the server side accesses the stored waypoints, and generates visually-oriented written driving directions, which are sent back to the client.
- the server side could also send graphical map data.
- the client receives the graphical map data, and requests any map tiles (or other image data) it doesn't already have displayed or cached.
- the client draws and displays the map, along with the visually-oriented driving directions.
- the client side can also be used to draw (overlay) the graphical driving directions, location markers, waypoints, etc. on the displayed map image.
Abstract
Digital mapping techniques are disclosed that provide visually-oriented information to the user, such as driving directions that include visual data points along the way of the driving route, thereby improving the user experience. The user may preview the route associated with the driving directions, where the preview is based on, for example, at least one of satellite images, storefront images, and heuristics and/or business listings.
Description
VISUALLY-ORIENTED DRIVING DIRECTIONS
ΓN DIGITAL MAPPING SYSTEM
RELATED APPLICATIONS
[0001] This application is a contimiation-in-part of U.S. Application No. 11/088,542, filed March 23, 2005, titled "Generating and Serving Tiles in a Digital Mapping System," which claims the benefit of U.S. Provisional Application Nos. 60/650,840, filed February 7, 2005, and 60/567,946, filed May 3, 2004, and 60/555,501, filed March 23, 2004. In addition, this application is related to U.S. Application No. 11/051,534, filed February 5, 2005, titled "A Digital Mapping System", which claims the benefit of U.S. Provisional Application Nos. 60/567,946, filed May 3, 2004, and 60/555,501, filed March 23, 2004. Each of these applications is herein incorporated in its entirety by reference. FIELD OF THE INVENTION
[0002] The invention relates to digital mapping systems, and more particularly, to computer generated driving directions that are visually-oriented. BACKGROUND OF THE INVENTION
[0003] Computerized mapping systems have been developed to search for, identify, and discover information about geographic locations. One form of such computerized mapping systems includes travel-planning Internet websites. With an excess of 50 million unique monthly users, such map sites are a very popular offering. Examples of such sites include AOL's MapQuest, Yahoo's Telcontar-based maps, and Microsoft's MapPoint.net suite. Such sites all work along the lines of a common model, as will now be described. [0004] When a Web user asks for a new map view (e.g., by entering a postal address, or by clicking a navigation link next to a current map view), the user's Web browser sends to a Web server a request indicating the boundaries of the new map view. The Web server in turn extracts the corresponding vector-based map data from a database, and draws a bitmap image of the map. The server then converts the bitmap to an image format supported by the user's Web browser and returns the image, sometimes embedded in HTML, to the user's Web
browser so that it can be displayed. Other map Web sites, such as Britain's MultiMaps or Australia's Wherels utilize a raster-based map database instead. In these cases, it is not necessary to extract vectors and draw a map image. Rather, these functions are replaced by simply extracting the appropriate part of a larger, pre-rendered image. [0005] Whether vector-based or raster-based, such existing map systems typically provide a computer-generated driving directions expressed in abstractions that are convenient for computers, such as "Stay on Main St. for 1.2 miles, and turn Right onto Center Street." Although such directions provide useful information that can be effectively employed to reach an intended destination, they can only be used in a literal sense. Other than the meaning of the words making up the directions, there is no further guidance to the user. [0006] What is needed, therefore, are digital mapping techniques that provide more information to the user.
SUMMARY OF THE INVENTION
[0007] One embodiment of the present invention provides a computer implemented method for generating visually-oriented driving directions. The method includes constructing a mapping between business listings and at least one of satellite images and storefront images of a target map area. The method continues with analyzing the at least one of satellite images and storefront images to identify waypoints, and evaluating distinctiveness of the identified waypoints, thereby providing a number of scored waypoints. The method further includes incorporating one or more of the scored waypoints into requested driving directions. The method may include applying OCR to storefront images to identify waypoints (e.g., to identify text and signage). The method may include performing visual recognition of chain stores using the at least one of satellite images and storefront images to identify waypoints (e.g., to recognize patterns of branding and architecture). The method may include performing non- visual recognition of chain stores using at least one of heuristics and business listings to identify waypoints (e.g., to recognize well-known and visually predictable businesses). The method may include storing the scored waypoints. In one such embodiment, the constructing, analyzing, evaluating, and storing are performed in advance of receiving requests for driving directions. Evaluating distinctiveness of the identified waypoints may include, for example, assigning a distinctiveness score based on at least one of a difference between a waypoint and its surroundings, scope of the difference, salience of the waypoint, visibility of the waypoint, and familiarity of pattern associated with the waypoint. The method may include adjusting a distinctiveness score of one or more of the
scored waypoints based on user feedback. Incorporating one or more of the scored waypoints into requested driving directions may further include serving those driving directions to the requestor. The method may include providing a visual preview of the driving directions using at least one of satellite images, storefront images, heuristics, and business listings. Incorporating one or more of the scored waypoints into requested driving directions may include selecting waypoints related to a destination of the requestor. One or more of the scored waypoints can be associated with a business advertising arrangement. In one such configuration, a cost-per-use of the one or more scored waypoints associated with a business advertisement is a factor that impacts waypoint distinctiveness scoring. [0008] Another embodiment of the present invention provides a computer implemented method for generating visually-oriented driving directions. In this configuration, the method includes constructing a mapping between business listings and at least one of satellite images and storefront images of a target map area, analyzing the at least one of satellite images and storefront images to identify waypoints, and evaluating distinctiveness of the identified waypoints, thereby providing a number of scored waypoints. The method further includes storing the scored waypoints for subsequent use in computer-generated driving directions. The method may include at least one of applying OCR to storefront images to identify waypoints, performing visual recognition of chain stores using the at least one of satellite images and storefront images to identify waypoints, and performing non- visual recognition of chain stores using at least one of heuristics and business listings to identify waypoints. Evaluating distinctiveness of the identified waypoints may include, for example, assigning a distinctiveness score based on at least one of a difference between a waypoint and its surroundings, scope of the difference, salience of the waypoint, visibility of the waypoint, and familiarity of pattern associated with the waypoint. The method may include adjusting a distinctiveness score of one or more of the scored waypoints based on user feedback. The method may include incorporating one or more of the scored waypoints into requested driving directions, providing a preview of the requested driving directions using the at least one of satellite images and storefront images, and serving the driving directions to the requestor. The method may include selecting one or more waypoints related to a destination of a user requesting driving directions, and incorporating the selected scored waypoints into the requested driving directions. One or more of the scored waypoints can be associated with a business advertising arrangement. In one such case, a cost-per-use of the one or more of
the waypoints associated with a business advertisement is a factor that impacts waypoint distinctiveness scoring.
[0009] Another embodiment of the present invention provides a computer implemented method for generating visually-oriented driving directions. In this configuration, the method includes identifying distinctive waypoints associated with an area covered by a digital mapping system, wherein the waypoints are visual data points along one or more driving routes and are in addition to road names (e.g., Main Street, RoutelOla, and Highway 1) and road topology (e.g., bridges, traffic circles, on-ramp/off-ramp, paved/unpaved, tolls). The method further includes incorporating one or more of the distinctive waypoints into computer-generated driving directions produced by the digital mapping system in response to a user request, thereby providing visually-oriented driving directions.
[0010] Each of these methods can be implemented, for example, using a tile-based mapping system. However, other systems, such as non-tile vector-based and raster-based mapping systems can be used to implement the methods as well, as will be apparent in light of this disclosure.
[0011] The features and advantages described herein are not all-inclusive and, in particular, many additional features and advantages will be apparent to one of ordinary skill in the art in view of the figures and description. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and not to limit the scope of the inventive subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 is a block diagram of a digital mapping system configured to provide visually-oriented driving directions, in accordance with one embodiment of the present invention.
[0013] Figure 2 is a block diagram of the location data serving system shown in Figure 1, configured for generating visually-oriented driving directions in accordance with one embodiment of the present invention.
[0014] Figure 3 is a block diagram of the waypoint identification and scoring module shown in Figure 2, configured for generating scored waypoints for use in visually-oriented driving directions, in accordance with one embodiment of the present invention. [0015] Figure 4 illustrates a method for generating visually-oriented driving directions, in accordance with one embodiment of the present invention.
DETAILED DESCRIPTION OF THE INVENTION
[0016] Digital mapping techniques are disclosed that provide visually-oriented information to the user, such as driving directions, thereby improving the user experience.
General Overview
[0017] As previously explained, computer- generated driving directions are typically expressed in abstractions that are convenient for computers, such as "Stay on Main St. for 1.2 miles", etc. hi contrast, human-generated directions tend to be more visually-oriented, such as "Stay on Main St, you'll pass a big Sears on your left, then turn right at the Dunldn Donuts". Through the use of satellite-based imagery and/or storefront images, a mapping system configured in accordance with an embodiment of the presenting invention can give more visually-oriented (and hence human-friendly) directions.
[0018] For instance, satellite-based imagery (such as that provided by Google Earth or Digital Globe) can be used to identify salient buildings, structures, and/or areas for use as way points (e.g., a particularly large building, an isolated building, a baseball or other large stadium, a park in an otherwise urban area). In addition, using image interrogation techniques, structures such as traffic lights and stop signs can be identified, which are also useful in giving directions.
[0019] Similarly, storefront (street-level) images can be used to identify salient buildings and features, such as stores with large visible logos, stores of unusual colors (e.g., "the bright purple store"), stores that are easily recognized because their brands are well-known and/or their store architectures and tradedress are distinctive (e.g., "the McDonalds"). This last category involving well-known brands, trademarks, tradedress, etc. can be identifiable even without image information (e.g., most everyone knows what a McDonalds looks like). Such visual data can be used to identify the target address, such as "Fred's Shoe Repair is in the middle of the block, just past the bright purple store".
[0020] The user can preview the driving route by a simulated drive-through or "fly- through" using the satellite/street-level images, in conjunction with the relevant digital maps. Thus, when the user actually drives to their targeted destination, the visual cues in the directions will remind the user of what to look for, and give that user a greater sense of confidence that he or she is on the right track. Numerous other benefits will be apparent in light of this disclosure.
System Architecture
[0021] Figure 1 is a block diagram of a tile-based digital mapping system configured in accordance with one embodiment of the present invention. As can be seen, the system includes a computing device 103 connected to a network 105. Various serving systems are also connected to the network 105, including a web serving system 110, a tile serving system 115, and a location data serving system 120. Other computing devices 103 (not shown) may also be in communication with the serving systems 110, 115, and 120 via the network 105. [0022] The computing device 103 may be any type of device configured for computing, such as a personal computer or laptop, a mobile phone, a personal digital assistant, a navigation system located in a vehicle, a handheld GPS system, and so on. The computing device 103 includes a browser (or other similar application) that allows the user to interface and communicate with other devices and systems on the network 105. Examples of browsers include Microsoft's Internet Explorer browser, Netscape's Navigator browser, Mozilla's Firefox browser, PalmSource's Web Browser, or any other browsing or application software capable of communicating with network 105. Generally stated, the computing device 103 can be any device that allows a user to access the serving systems 110, 115, and 120 via the network 105.
[0023] The web serving system 110 is the part of an overall system that delivers the initial HTML (hypertext markup language), images, scripting language (e.g., JavaScript, JScript, Visual Basic Script), XSLT (extensible stylesheet language transformation), and other static elements that are used by the browser on computing device 103. Note that this serving system 110 may include one or more servers operating under a load balancing scheme, with each server (or a combination of servers) configured to respond to and interact with the computing device 103. The web serving system 110 can be implemented with conventional technology, as will be apparent in light of this disclosure.
[0024] The tile serving system 115 is the part of the overall system responsible for delivering individual map tiles in response to requests from the computing device 103, with each tile being uniquely defined, for example, by x, y and z values that coordinates to an overall tile-based map. Other tile identification schemes can be used as well. The tile serving system 115 may include one or more servers operating under a load balancing scheme, with each server (or a combination of servers) configured to respond to and interact with the computing device 103. Example architecture and functionality of the tile serving system 115 is further discussed in the previously incorporated U.S. Application No. 11/088,542.
[0025] The location data serving system 120 is the part of the overall system that delivers location data of various forms to the computing device 103. Its functions include, for example, finding the geographic location of a street address, generating and formatting visually-oriented driving directions, and searching for location-specific results to a query (e.g., as with the Google Local Search service). Other services may also be provided. In general, when the user enters a search string, it is put into a request by the computing device 103, and sent to the location data serving system 120 via the network 105. The location data serving system 120 then determines what the request is for (e.g., generate driving directions with visual "fly-through" of route, or generate visually-oriented driving directions, or generate both visually-oriented driving directions and visual "fly-through" of route), and responds with the appropriate data from various sub-systems, such as geo-coders, routing engines, and local search indexes or databases (e.g., including a scored waypoint database), in a format that computing device 103 can use to present the data to the user (e.g., via a browser). Example architecture and functionality of the location data serving system 120 will be discussed in turn with reference to Figures 2-4.
[0026] The network 105 may be any type of communications network, such as a local area network (e.g., intranet), wide area network (e.g., internet), or some combination thereof. Alternatively, the network 105 may be a direct connection between the computing device 103 and the serving systems 110, 115, and 120. In general, the computing device 103, network 105, and/or serving systems 110, 115, and 120 may be in communication via any type of wired or wireless connection, using a wide variety of communication protocols.
Location Data Serving System
[0027] Figure 2 is a block diagram of the location data serving system 120 shown in Figure 1, configured for generating visually-oriented driving directions in accordance with one embodiment of the present invention. As can be seen, the system 120 includes a waypoint identification and scoring module 205, a scored waypoint database 210, and a driving direction generator 215.
[0028] As previously discussed, the location data serving system 120 can be part of the overall digital mapping system that delivers mapping data of various forms (e.g., location data, map images, etc) to the computing device 103. In this particular embodiment, the location data is in the form of visually-oriented driving directions. Note that these driving directions can be provided in conjunction with a map that includes an overlay of the driving directions in graphical form as well. For instance, in the embodiment shown in Figure 1, the
digital mapping system is configured to provide tile-based maps, as explained in detail in the previously incorporated U.S. Application Nos. 11/088,542 and 11/051,534. However, the present invention is not intended to be limited to systems that provided tile-based maps. Rather, embodiments of the present invention can also be used with other systems, such as non-tile vector-based and raster-based mapping systems. Likewise, other embodiments of the present invention can be used to provide visually-oriented driving directions (with no map). Numerous mapping system configurations can be used. The description herein focuses on the visually-oriented driving directions component of the system. Other components and map system functionality not shown or otherwise discussed, whether conventional or custom, can be used as desired in conjunction with visually-oriented driving directions, as will be apparent in light of this disclosure.
[0029] In the embodiment shown in Figure 2, the location data serving system 120 operates in both an off-line capacity, as well as in an on-line capacity. In particular, "scored waypoints" are generated by the waypoint identification and scoring module 205 in an offline process, and stored in the scored waypoint database 210. This off-line process effectively stocks the database 210 with waypoints and their corresponding distinctiveness scores. During on-line operations, requests (e.g., HTTP) for driving directions are received by the driving direction generator 215. The request can be initiated, for example, by a user engaging a web browser of a computing device 103 to access the system. In response to such a client request, the driving direction generator 215 accesses the scored waypoint database 210 and integrates the relevant scored waypoints into the driving directions generated in response to the request. These visually-oriented driving directions are then served to the requestor via the network 105 and web browser of the computing device 103. [0030] A waypoint is a visual data point along the way of a driving route, such as a building, business, park, billboard, or some other visually noticeable structure or place. The distinctiveness of the waypoint is indicative of its visual prominence. Color (e.g., purple), shape (e.g., round building), size (e.g., 7 mile long park), degree of landmark recognition (e.g., Golden Gate Bridge), and/or degree of brand recognition (e.g., Golden Arches) are example factors that can be used to evaluate the distinctiveness of a waypoint. [0031] In addition to the written driving directions, the digital mapping system in which the location data serving system 120 operates may also generate and serve graphical map data relevant to the client request. In one such embodiment, the client receives the requested graphical map data, and requests any map tiles it doesn't already have displayed or cached
(e.g., as explained in the previously incorporated U.S. Application No. 11/051,534. When the tiles are received from the server side of the digital mapping system (e.g., from the tile serving systems 115), the client draws and displays the map, along with the visually-oriented driving directions. The client side can also be used to draw (e.g., overlay) the graphical driving directions, location markers, etc on the map image. Note here that the distinctive waypoints can also be overlaid on the map, along with the route itself.
Waypoint Identification and Scoring
[0032] Figure 3 is a block diagram of the waypoint identification and scoring module 205 shown in Figure 2, configured for generating scored waypoints for use in visually-oriented driving directions, in accordance with one embodiment of the present invention. The module 205 generates scored waypoints, which can then be stored in scored waypoint database 210. These scored waypoints can then be accessed by a driving direction generator to create visually-oriented driving directions.
[0033] As can be seen, the module 205 includes a storefront image database 305, a satellite image database 310, a yellow pages database 315, an image registration module 320, a satellite image processor 325, a storefront image process 330, an optical character recognition (OCR) module 335, a non-visual recognition module 340, a visual recognition module 345, and a waypoint distinctiveness evaluation module 350. Numerous variations on this configuration for generating scored waypoints will be apparent in light of this disclosure, and the present invention is not intended to be limited to any one such embodiment. [0034] In operation at preprocessing time (off-line), the waypoint identification and scoring module 205 employs one or more a databases of images (e.g., storefront image database 305 and satellite image database 310), together with a corresponding database of business listings (e.g., yellow pages database 315). Each database can be structured to facilitate efficient access of data, and include various types of information. For example, each street-level image (e.g., digital photograph taken using a GPS-enable camera) stored in the storefront image database 305 can be indexed by geocode, and associated with corresponding GPS coordinates. Likewise, each satellite image stored in the satellite image database 310 can be indexed by geocode, and associated with corresponding GPS coordinates. The yellow pages database 315 can be structured, for example, as conventionally done.
[0035] In an alternative embodiment, the illustrated databases are integrated into a single database. Also, other databases or information sets could be included, such as a conventional white pages database or other such listing service databases. Further note that the image
databases may include multiple views and/or zoom levels of each targeted area. For instance, one storefront image can be taken from an angle as it would be seen coming down one direction of the street, while another storefront image of the same address could be taken from an angle as it would be seen coming down the other direction of the street. Thus, depending on the driving directions, either image could be used.
[0036] The image registration module 320 is programmed or otherwise configured to construct a mapping between images and business listings. In one embodiment, this mapping is accomplished by a combination of image segmentation using standard image-processing techniques (e.g., edge detection, etc.) and interpolation of a business's street address within the range of street addresses known to be contained in the image. Image registration is done for both storefront images and satellite images. The mapping can be implemented, for example, with a pointer or address scheme that effectively connects images from an image database to listings in the yellow pages database. Alternatively, a single database can be built as the image registration process is carried out, where the records of the single database are indexed by geocode, and each record includes image data and related yellow page listing information.
[0037] In the embodiment shown, image processing is performed by accessing the images by way of the image registration module 320 (e.g., which can access the images stored in the respective databases 305 and 310 using a pointer or addressing scheme). Other embodiments can access the images directly from their respective databases. In any case, image processing is performed to propose candidate waypoints based on detected visual features. [0038] In more detail, the satellite image processor 325 is programmed or otherwise configured to recognize navigational features included in the satellite images. Certain navigational features are particularly useful for drivers, because a driver has to attend to such features anyway, including the likes of traffic lights, stop signs, on/off-ramps, traffic circles, city blocks, tunnels, bridges, end of divided highway, and other driver-centric features. These navigational features facilitate human directions such as: "turn right at the third light", "get off at the second off-ramp", and "go through 2 traffic circles and then it's on your right." [0039] In one particular embodiment, the satellite image processor 325 uses standard machine-learning techniques to learn to recognize these navigational features from the satellite images. In one such approach, humans annotate a set of satellite images with instances of the desired feature (e.g., all traffic circles), and to then use standard machine- learning algorithms (e.g., SVMs, boosting over decision stumps, decision lists, etc.) to learn
image features that correctly predict the occurrence of that feature. Once trained, the machine-learning algorithms can be used to recognize the learned or known navigational features in all the satellite images. These identified features can then be evaluated for use as waypoints.
[0040] In addition to learning from satellite images, many navigational features can be learned from GPS/accelerometer data associated with the storefront (street-level) images, as will be explained in turn.
[0041] The storefront image processor 330 is programmed or otherwise configured to analyze storefront images. In one embodiment, this analysis is carried out at both a coarse level (e.g., width, height, color histograms) and a more refined level (e.g., segmentation into facade, doors, windows, roof; architectural elements such as pillars and balconies; decorative elements such as awnings, signage, neon lights, painted designs). Such analysis can carried out, for example, using standard image-processing techniques (e.g., computer vision). Standard feature extraction algorithms typically extract high level information from images, such as shapes, colors, etc. Pattern recognition algorithms can then be applied to classify the extracted information so as to "recognize" objects in the storefront images. [0042] hi addition to learning navigational features from satellite images, many navigational features can be learned from GPS/accelerometer data associated with the storefront images taken at the street-level. For instance, when collecting storefront images (e.g., using a digital camera in a car that is equipped with GPS receiver and an accelerometer), GPS and 3-D accelerometer readings could also be collected for each photographic image taken. Certain navigational features are associated with a particular set of associated readings. For example: on-ramps would have strong acceleration, and off- ramps would have strong deceleration; and traffic lights would have deceleration to zero, followed at regular intervals by acceleration. The GPS could be used to record where these acceleration/deceleration patterns occurred. Once the position of these navigational features (e.g., on-ramps, off-ramps, and lights) are learned, they can be used in giving driving directions.
[0043] All of these identified features can then be evaluated for use as waypoints. For instance, waypoints can be proposed based on observed locally distinctive or otherwise salient features, including any measurable dimension or highly noticeable quality (e.g., unusually wide/narrow/tall/short building, building with purple door, orange roof, green neon sign, etc). If a feature is "locally distinctive", it is unique within some vicinity, such as the
only pink building on the block. The features discussed so far are intrinsic to the building, but extrinsic features can also be used (e.g., the building at the corner, the first building over the railroad tracks, etc).
[0044] While the storefront image processor 330 can be used to detect decorative and structural features within the images, the OCR module 335 can be applied to the storefront images to read the storefront signage (if any), using standard OCR algorithms and techniques. This text recognition enables waypoint proposal of visual features such as "the bar with the neon Karaoke sign in the window." In one particular embodiment, the OCR accuracy of module 335 can be improved by identifying what kind of a store is in the storefront image, based on its corresponding category listing (e.g., bars and restaurants) in the yellow pages database 315. Recall that the image registration module 320 has already mapped the images to corresponding listings within the yellow pages database 315, thereby facilitating this context identification for the OCR process. In addition, text related to that yellow pages category can be obtained, for example, by visiting web sites of stores in that category, and adjusting the language model used for OCR module 335, accordingly. This supplemental information from yellow page listings and/or websites enables the OCR module 335 to be informed of the context in which it is operating.
[0045] The visual learning module 345 receives the proposed waypoint information output by each of the image processors 325 and 330, and the OCR module 335. The visual learning module 345 is programmed or otherwise configured to perform visual learning based on established knowledge. For instance, in one particular embodiment, the visual learning module 345 recognizes chain stores. Chain stores (e.g., McDonalds) are a common element in human directions.
[0046] In particular, the visual learning module 345 can learn which chain stores have distinctive logos/buildings by, for example, identifying common elements across storefront images of multiple branches using the results of the storefront image processor 325 and OCR module 335, combined with yellow page listings and standard translation/rotation/scaling transformations to align multiple images, and even simpler image-processing techniques such as orientation histograms.
[0047] For example, the visual learning module 345 would discover that McDonalds frequently have the same logo (e.g., the word "McDonalds" in a particular color and font), a glass storefront, Golden Arches, etc. In this sense, the visual learning module 345 learns a prototype of what the target chain store typically looks like. In addition, the visual learning
module 345 identifies the extent to which each branch of the chain store at a given address matches the prototype. If a McDonalds branch at a given address sufficiently matches the common McDonalds' features found in the prototype, that McDonalds branch is a useful waypoint to use in driving directions, because it will be easily recognized by humans as a McDonalds. The combination of these two features (prototyping and comparison to prototype) improves the system's reliability in finding useful waypoints. Note that a given chain-store branch may be actually be a "bad" waypoint if, for instance, it doesn't match the prototype well. For example, a McDonalds in an upscale neighborhood may be required to conform to the prevailing architecture (e.g., no large Golden Arches allowed), thus reducing its recognizability as a McDonalds. Likewise, a given chain-store branch may actually be a "bad" waypoint if, for instance, that particular branch is not clearly visible from the road (e.g., obscured by foliage or other buildings). The visual learning module 345 will avoid using such branches of the chain store as waypoints, thereby further refining results from the image processors 325 and 330, and the OCR module 335.
[0048] Practically speaking, McDonalds is well-known and fairly consistent in its appearance (even when restricted by local ordinance), and may not need to be processed by the visual recognition module (e.g., McDonalds image and address data could simply pass through the visual learning module 345 as a proposed waypoint for a given area). Note that the non-visual recognition module 340 can be used to identify McDonalds and other well- known architectures, as will be apparent in light of this disclosure. On the other hand, a more regional restaurant chain that is less well-known, but fairly consistent in its appearance, could be identified as a waypoint of comparable efficacy to McDonalds by the visual learning module 345.
[0049] Unlike the visual learning module 345, the non-visual recognition module 340 can be used to recognize chain stores (and other such consistent structures and the like) without reference to any images. Rather, only address information (e.g., from the yellow pages database 315 in this embodiment) is needed. For instance, instead of learning from images of multiple branches of a chain store, heuristics about store recognizability can be used. [0050] For example, gas stations are often good, recognizable waypoints because they tend to display their logo prominently, and they have distinctive architecture (e.g., canopy over gas pumps and ample signage). Likewise, stores with a multitude of branches (e.g., over 1000) in the yellow pages tend to have distinctive branding and widespread marketing, and are typically well-recognized. Similarly, chain grocery stores often have large logos, large
buildings, and large parking lots. In addition, they are typically well-known in the communities in which they exist.
[0051] hi any such cases, the non-visual recognition module 340 uses address listings to identify chain stores and other predictable/consistent architectures that are known to be distinctive, and proposes them as waypoints.
[0052] The waypoint distinctiveness evaluation module 350 evaluates the distinctiveness of the proposed waypoints received from the non-visual recognition module 340 and the visual recognition module 345. The distinctiveness measure can range, for example, from 1 to 10, and can take into account a number of aspects, where a distinctiveness score of 1 is indicative of a low distinctiveness rating, and a distinctiveness score of 10 is indicative of a high distinctiveness rating, hi one particular embodiment, one or more of the following factors are considered in assigning a distinctiveness score: the magnitude of difference of the waypoint and its surroundings such as neighbors and open space (e.g., bright red is far away from white in color-space); the scope of the difference (e.g., the only bright red building on an entire block is more distinctive than a bright red building with another bright red building 2 doors down); the salience of a waypoint (e.g., a bright red door is less salient than a bright red entire facade); the visibility of the waypoint (e.g., a building separated from its neighbors by an empty lot may be easier to spot than a store in a row of connected buildings); the familiarity of the pattern associated with the waypoint (e.g., in the case of chain stores that have very familiar branding).
Incorporating Wavpoints into Driving Directions
[0053] Having identified waypoints and their distinctiveness scores, they can be used in several ways when generating driving directions (e.g., using the driving directions generator 215). For instance, turns can be identified using waypoints at or near an intersection (e.g., "you'll see a McDonalds on your right and a Mobil station on your left" or "turn right just past the Home Depot on your right"), hi one particular embodiment, the waypoints that are given are on the same side of the street as the turn (e.g., waypoints on the right-hand side for right turns), so drivers can focus on one side of the road at a time. Confirmatory waypoints can be provided, such as "You'll pass a Safeway on your left, then a large park on your right." Also, "gone too far" waypoints can be given, such as "If you see a Tower Records on your right, you've gone too far." Waypoints can also be used as early warning indicators, to signal the driver to start watching for a turn, especially if the driver has been on the same road for awhile ("you'll be on Center St. for 20 miles; after you pass the Crab tree Shopping Center on
your left, start watching for your left turn"). Waypoints can also be used to identify the target destination (e.g., "your destination is the three-story white house on the left" or "you'll see the Fry's on your right; look for the big red logo"). The entire route can be chosen with the quality of waypoints in mind. For instance, the user can be directed along a somewhat longer route if the longer path affords better waypoints. In one particular embodiment, the user can be offered a choice of maximizing "ease of following directions" versus "shortest distance" (or other such alternatives).
[0054] In all of these uses of waypoints, there is a balance between choosing a waypoint in exactly the desired location with choosing a waypoint with a higher distinctiveness score. For example, it may be better to direct the user as follows: "Your destination is the third house after the big red building" compared to: "Your destination is the house on the left with a brown front door." Note also that streets can be used as waypoints (e.g., "It's the first right- hand turn after you cross Maple St."). In such an embodiment, the waypoint distinctiveness evaluation module 350 is configured to score the salience of streets (e.g., "large street with two lanes in each directions" or "a boulevard with grassy median").
[0055] The driving directions can be generated using conventional technology or as described in the previously incorporated U.S. Application Nos. 11/088,5.42 and 11/051,534, with the addition of the incorporation of one or more of the scored waypoints. m one embodiment, the waypoints are incorporated into a pre-selected set of driving directions, where conventional driving directions are generated, and then effectively annotated with waypoints. These waypoints can be added to the driving directions as text (as in the previously given examples) and/or as images (e.g., McDonalds icon). As will be apparent in light of this disclosure, driving directions can also be generated for a plurality of routes, and taking the waypoint scores along each route into account when selecting the best route to give.
User Feedback
[0056] In one embodiment, a mechanism for collecting feedback from users as to the utility of different waypoints is provided. For example, a GPS record of the user's actual route taken when attempting to follow the served driving directions (e.g., from a handheld GPS receiver/recorder, or an on-board GPS navigation system, or a GPS-enabled cellphone). The reporting of GPS data can be transparent to the user, or enabled by the user. [0057] In one particular embodiment, a user feedback module is downloaded into the user's browser, and is programmed to receive the GPS data and report it back to the serving system.
An alternate feedback approach that can be used is to provide a form (e.g., online webform or email, and/or paper/fax submissions) for users to fill out and send back with positive/negative feedback about each step of the visually-oriented driving directions. The user feedback could be used, for example, by the waypoint distinctiveness evaluation module 350 to reduce the distinctiveness score of waypoint(s) that were used to indicate a turn that the user missed, or to increase the distinctiveness score of waypoint(s) that were used to indicate a turn that the user found.
[0058] Feedback from multiple users could be categorized and tallied by waypoint, so that a meaningful sample of feedback data could be analyzed when upgrading/downgrading the distinctiveness score of a waypoint, thereby eliminating or otherwise reducing the impact of anomalies or individual biases regarding a particular waypoint. Numerous statistical techniques can be used to properly value the user feedback.
Advertising;
[0059] In one particular embodiment, businesses are allowed to bid or otherwise pay to be included as a waypoint. The "cost-per-use" of a waypoint could then be an additional factor that would be taken into account when scoring each waypoint. Furthermore, incorporating one or more of the scored waypoints into requested driving directions could include selecting waypoints related to a destination of the requestor. For instance, if the user is asking for driving directions to a national park, then waypoints such as camping equipment stores could be selected for integration into the served driving directions. This choice of waypoints has several benefits: (1) the waypoints might be of interest in their own right; (2) the waypoints are more likely to be familiar to the user and thus better waypoints; and (3) advertisers would be likely to pay more for such targeted waypoint usage.
Previews
[0060] In one embodiment, fly-throughs (e.g., using satellite images) or walkthroughs (e.g., using storefront images) of the user's route could be provided, in addition to the visually- oriented driving directions. This would better prepare the user visually for following the directions, as they would be able to see the waypoints referenced in the driving directions ahead of time. In addition, photos of the waypoints and intersections could be printed in advance by the user to aid the user in spotting them when driving the actual route. [0061] In one particular embodiment, an "autoplay preview" feature is provided on the client side. When selected by the user, a request for preview data would be sent to the server side. The client could initially be given the entire route, which can be represented, for
example, as a sequence of latitude/longitude pairs of line segments approximating the path. The client can then use simple linear interpolation to follow the route. For instance, the client could display a map (e.g., satellite map or bitmap map) centered initially at the starting point of the route, with a progress marker (e.g., a car icon), and would continuously scroll the display along the desired route. Movement of the progress marker could be carried out using script code (e.g., JavaScript) executing on the client. The client could pre-fetch map tiles (or other map image) in conjunction with moving the progress marker to create an uninterrupted viewing experience.
[0062] An analogous pre-fetching scheme could be used for displaying storefront images along the route, which may be more meaningful to the user, given the ease of perceiving street-level images. A perspective transformation can be used to show the storefronts on both sides of the street as the user drives (or walks) along the route. Other elements can be added to the preview as desired, such as traffic lights, street signs, and even simulated other cars, etc., to enhance realism. In one such embodiment, the user is able to control the playback (e.g., via start/stop/pause/reverse control objects included in the user interface). [0063] In one particular configuration, the user is allowed to divert from the original route, so as to explore alternate routes and the surrounding area ("virtual reconnaissance"). Note in this latter case, however, that pre-fetching becomes more difficult. To reduce latency of such real-time image delivery, predictive caching of the relevant map images/storefront images can be used based on the user's current direction and previous use habits. A number of predictive caching and image delivery schemes can be used, as will be apparent in light of this disclosure.
Methodology
[0064], Figure 4 illustrates a method for generating visually-oriented driving directions, in accordance with one embodiment of the present invention. The method can be carried out, for example, using the systems of Figures 1-3. However, any mapping system can integrate the methodology.
[0065] As can be seen, the method includes an off-line portion and an on-line portion. The off-line portion includes constructing 405 a mapping between business listings and satellite and storefront images of the target map area. The method continues with analyzing 410 the satellite images to identify waypoints, and analyzing 415 the storefront images to identify waypoints. The method further includes applying 420 OCR to storefront images to identify
waypoints. As previously explained, image analysis can be used to detect decorative and structural features within the images, and OCR can be used to detect the storefront signage. [0066] The off-line portion of the method continues with performing 425 visual recognition of chain stores using images (e.g., storefront and satellite) to identify waypoints. As previously explained, the yellow pages (or other such listings) can be used to identify the addresses of all locations a particular chain store, and then images of those locations can be analyzed to identify the common elements across the images. Chain stores with such consistent elements can be used as waypoints, as they are likely to be recognized. [0067] The off-line portion of the method also includes performing 430 non- visual recognition of chain stores using heuristics and/or business listings to identify waypoints. As previously explained, there are many rules of thumb that can be applied to identify waypoints, without using image data. In addition, both visual (e.g., machine learning) and non-visual (e.g., GPS/acceleration data collection) techniques can be used to identify navigational features, which can also be used as waypoints.
[0068] The method continues with evaluating 435 the distinctiveness of the proposes waypoints. The distinctiveness measure can range, for example, from 1 (low distinctiveness) to 10 (high distinctiveness), and takes into account a number of factors as previously explained. Numerous rating schemes are possible, and the end result is a number of scored waypoints. The off-line portion of the method further includes storing 440 the scored waypoints
[0069] The on-line portion of the method includes incorporating 445 one or more scored waypoints into requested driving directions. As previously explained, when a request for written driving directions is received from a client, a driving direction generator on the server side accesses the stored waypoints, and generates visually-oriented written driving directions, which are sent back to the client. In addition to the written driving directions, the server side could also send graphical map data. In one particular embodiment, the client receives the graphical map data, and requests any map tiles (or other image data) it doesn't already have displayed or cached. When the tiles are received from the server side, the client draws and displays the map, along with the visually-oriented driving directions. The client side can also be used to draw (overlay) the graphical driving directions, location markers, waypoints, etc. on the displayed map image.
[0070] The foregoing description of the embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit
the invention to the precise form disclosed. Many modifications and variations are possible in light of this disclosure. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto.
Claims
What is claimed is: 1. A computer implemented method for generating visually-oriented driving directions, comprising: constructing a mapping between business listings and at least one of satellite images and storefront images of a target map area; analyzing the at least one of satellite images and storefront images to identify waypoints; evaluating distinctiveness of the identified waypoints, thereby providing a number of scored waypoints; and incorporating one or more of the scored waypoints into requested driving directions.
2. The method of claim 1 further comprising: applying OCR to storefront images to identify waypoints.
3. The method of claim 1 further comprising: performing visual recognition of chain stores using the at least one of satellite images and storefront images to identify waypoints.
4. The method of claim 1 further comprising: performing non- visual recognition of chain stores using at least one of heuristics and business listings to identify waypoints.
5. The method of claim 1 further comprising: storing the scored waypoints.
6. The method of claim 5 wherein the constructing, analyzing, evaluating, and storing are performed in advance of receiving requests for driving directions.
7. The method of claim 1 wherein evaluating distinctiveness of the identified waypoints includes assigning a distinctiveness score based on at least one of a difference between a waypoint and its surroundings, scope of the difference, salience of the waypoint, visibility of the waypoint, and familiarity of pattern associated with the waypoint.
8. The method of claim 1 further comprising: adjusting a distinctiveness score of one or more of the scored waypoints based on user feedback.
9. The method of claim 1 wherein incorporating one or more of the scored waypoints into requested driving directions further includes serving those driving directions to the requestor.
10. The method of claim 1 further comprising: providing a visual preview of the driving directions using at least one of satellite images, storefront images, heuristics, and business listings.
11. The method of claim 1 further wherein incorporating one or more of the scored waypoints into requested driving directions includes selecting waypoints related to a destination of the requestor.
12. The method of claim 1 wherein one or more of the scored waypoints are associated with a business advertising arrangement.
13. The method of claim 12 wherein a cost-per-use of the one or more scored waypoints associated with a business advertisement is a factor that impacts waypoint distinctiveness scoring.
14. The method of claim 1 wherein the method is implemented using a tile-based mapping system.
15. A computer implemented method for generating visually-oriented driving directions, comprising: constructing a mapping between business listings and at least one of satellite images and storefront images of a target map area; analyzing the at least one of satellite images and storefront images to identify waypoints; applying OCR to storefront images to identify waypoints; performing visual recognition of chain stores using the at least one of satellite images and storefront images to identify waypoints; performing non- visual recognition of chain stores using at least one of heuristics and business listings to identify waypoints; evaluating distinctiveness of the identified waypoints, thereby providing a number of scored waypoints; storing the scored waypoints; and ' incorporating one or more of the scored waypoints into requested driving directions.
16. The method of claim 15 wherein the method is implemented using a tile-based mapping system.
17. A computer implemented method for generating visually-oriented driving directions, comprising: constructing a mapping between business listings and at least one of satellite images and storefront images of a target map area; analyzing the at least one of satellite images and storefront images to identify waypoints; evaluating distinctiveness of the identified waypoints, thereby providing a number of scored waypoints; and storing the scored waypoints for subsequent use in computer-generated driving directions.
18. The method of claim 17 further comprising at least one of: applying OCR to storefront images to identify waypoints; performing visual recognition of chain stores using the at least one of satellite images and storefront images to identify waypoints; and performing non- visual recognition of chain stores using at least one of heuristics and business listings to identify waypoints.
19. The method of claim 17 wherein evaluating distinctiveness of the identified waypoints includes assigning a distinctiveness score based on at least one of a difference between a waypoint and its surroundings, scope of the difference, salience of the waypoint, visibility of the waypoint, and familiarity of pattern associated with the waypoint.
20. The method of claim 17 further comprising: adjusting a distinctiveness score of one or more of the scored waypoints based on user feedback.
21. The method of claim 17 further comprising: incorporating one or more of the scored waypoints into requested driving directions; providing a visual preview of the requested driving directions using at least one of satellite images, storefront images, heuristics, and business listings; and serving the driving directions to the requestor.
22. The method of claim 17 further comprising: selecting one or more waypoints related to a destination of a user requesting driving directions; and incorporating the selected scored waypoints into the requested driving directions.
23. The method of claim 17 wherein one or more of the scored waypoints are associated with a business advertising arrangement.
24. The method of claim 23 wherein a cost-per-use of the one or more of the waypoints associated with a business advertisement is a factor that impacts waypoint distinctiveness scoring.
25. The method of claim 17 wherein the method is implemented using a tile-based mapping system.
26. A computer implemented method for generating visually-oriented driving directions, comprising: identifying distinctive waypoints associated with an area covered by a digital mapping system, wherein the waypoints are visual data points along one or more driving routes and are in addition to road names and road topology; and incorporating one or more of the distinctive waypoints into computer-generated driving directions produced by the digital mapping system in response to a user request, thereby providing visually-oriented driving directions.
27. The method of claim 26 wherein the digital mapping system is a tile-based mapping system.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/181,386 US7831387B2 (en) | 2004-03-23 | 2005-07-13 | Visually-oriented driving directions in digital mapping system |
US11/181,386 | 2005-07-13 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2007008809A2 true WO2007008809A2 (en) | 2007-01-18 |
WO2007008809A3 WO2007008809A3 (en) | 2008-01-31 |
Family
ID=37637834
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2006/026745 WO2007008809A2 (en) | 2005-07-13 | 2006-07-10 | Visually-oriented driving directions in digital mapping system |
Country Status (2)
Country | Link |
---|---|
US (2) | US7831387B2 (en) |
WO (1) | WO2007008809A2 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8219316B2 (en) | 2008-11-14 | 2012-07-10 | Google Inc. | System and method for storing and providing routes |
US8489326B1 (en) | 2010-02-09 | 2013-07-16 | Google Inc. | Placemarked based navigation and ad auction based on placemarks |
US8754777B1 (en) | 2010-08-13 | 2014-06-17 | Google Inc. | System and method for predicting user route and destination |
Families Citing this family (168)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR100571429B1 (en) * | 2003-12-26 | 2006-04-17 | 한국전자통신연구원 | Method of providing online geometric correction service using ground control point image chip |
JP4722433B2 (en) * | 2004-08-25 | 2011-07-13 | アルパイン株式会社 | Car navigation system |
US8838591B2 (en) | 2005-08-23 | 2014-09-16 | Ricoh Co., Ltd. | Embedding hot spots in electronic documents |
US8600989B2 (en) * | 2004-10-01 | 2013-12-03 | Ricoh Co., Ltd. | Method and system for image matching in a mixed media environment |
US7639387B2 (en) | 2005-08-23 | 2009-12-29 | Ricoh Co., Ltd. | Authoring tools using a mixed media environment |
US8385589B2 (en) | 2008-05-15 | 2013-02-26 | Berna Erol | Web-based content detection in images, extraction and recognition |
US9373029B2 (en) | 2007-07-11 | 2016-06-21 | Ricoh Co., Ltd. | Invisible junction feature recognition for document security or annotation |
US7812986B2 (en) | 2005-08-23 | 2010-10-12 | Ricoh Co. Ltd. | System and methods for use of voice mail and email in a mixed media environment |
US8005831B2 (en) * | 2005-08-23 | 2011-08-23 | Ricoh Co., Ltd. | System and methods for creation and use of a mixed media environment with geographic location information |
US7991778B2 (en) | 2005-08-23 | 2011-08-02 | Ricoh Co., Ltd. | Triggering actions with captured input in a mixed media environment |
US8856108B2 (en) | 2006-07-31 | 2014-10-07 | Ricoh Co., Ltd. | Combining results of image retrieval processes |
US8195659B2 (en) | 2005-08-23 | 2012-06-05 | Ricoh Co. Ltd. | Integration and use of mixed media documents |
US8144921B2 (en) | 2007-07-11 | 2012-03-27 | Ricoh Co., Ltd. | Information retrieval using invisible junctions and geometric constraints |
US9405751B2 (en) | 2005-08-23 | 2016-08-02 | Ricoh Co., Ltd. | Database for mixed media document system |
US8156427B2 (en) | 2005-08-23 | 2012-04-10 | Ricoh Co. Ltd. | User interface for mixed media reality |
US8332401B2 (en) | 2004-10-01 | 2012-12-11 | Ricoh Co., Ltd | Method and system for position-based image matching in a mixed media environment |
US7551780B2 (en) * | 2005-08-23 | 2009-06-23 | Ricoh Co., Ltd. | System and method for using individualized mixed document |
US8276088B2 (en) | 2007-07-11 | 2012-09-25 | Ricoh Co., Ltd. | User interface for three-dimensional navigation |
US7669148B2 (en) | 2005-08-23 | 2010-02-23 | Ricoh Co., Ltd. | System and methods for portable device for mixed media system |
US7702673B2 (en) | 2004-10-01 | 2010-04-20 | Ricoh Co., Ltd. | System and methods for creation and use of a mixed media environment |
US8510283B2 (en) | 2006-07-31 | 2013-08-13 | Ricoh Co., Ltd. | Automatic adaption of an image recognition system to image capture devices |
US9530050B1 (en) | 2007-07-11 | 2016-12-27 | Ricoh Co., Ltd. | Document annotation sharing |
US8825682B2 (en) | 2006-07-31 | 2014-09-02 | Ricoh Co., Ltd. | Architecture for mixed media reality retrieval of locations and registration of images |
US7970171B2 (en) | 2007-01-18 | 2011-06-28 | Ricoh Co., Ltd. | Synthetic image and video generation from ground truth data |
US7885955B2 (en) * | 2005-08-23 | 2011-02-08 | Ricoh Co. Ltd. | Shared document annotation |
US9384619B2 (en) | 2006-07-31 | 2016-07-05 | Ricoh Co., Ltd. | Searching media content for objects specified using identifiers |
US8156115B1 (en) | 2007-07-11 | 2012-04-10 | Ricoh Co. Ltd. | Document-based networking with mixed media reality |
US8156116B2 (en) | 2006-07-31 | 2012-04-10 | Ricoh Co., Ltd | Dynamic presentation of targeted information in a mixed media reality recognition system |
US7672543B2 (en) * | 2005-08-23 | 2010-03-02 | Ricoh Co., Ltd. | Triggering applications based on a captured text in a mixed media environment |
US8184155B2 (en) | 2007-07-11 | 2012-05-22 | Ricoh Co. Ltd. | Recognition and tracking using invisible junctions |
US8521737B2 (en) | 2004-10-01 | 2013-08-27 | Ricoh Co., Ltd. | Method and system for multi-tier image matching in a mixed media environment |
US9171202B2 (en) | 2005-08-23 | 2015-10-27 | Ricoh Co., Ltd. | Data organization and access for mixed media document system |
US7587412B2 (en) * | 2005-08-23 | 2009-09-08 | Ricoh Company, Ltd. | Mixed media reality brokerage network and methods of use |
US8086038B2 (en) | 2007-07-11 | 2011-12-27 | Ricoh Co., Ltd. | Invisible junction features for patch recognition |
US8335789B2 (en) | 2004-10-01 | 2012-12-18 | Ricoh Co., Ltd. | Method and system for document fingerprint matching in a mixed media environment |
US8868555B2 (en) | 2006-07-31 | 2014-10-21 | Ricoh Co., Ltd. | Computation of a recongnizability score (quality predictor) for image retrieval |
US7917554B2 (en) | 2005-08-23 | 2011-03-29 | Ricoh Co. Ltd. | Visibly-perceptible hot spots in documents |
US8176054B2 (en) | 2007-07-12 | 2012-05-08 | Ricoh Co. Ltd | Retrieving electronic documents by converting them to synthetic text |
US8369655B2 (en) | 2006-07-31 | 2013-02-05 | Ricoh Co., Ltd. | Mixed media reality recognition using multiple specialized indexes |
US8949287B2 (en) | 2005-08-23 | 2015-02-03 | Ricoh Co., Ltd. | Embedding hot spots in imaged documents |
US7920759B2 (en) | 2005-08-23 | 2011-04-05 | Ricoh Co. Ltd. | Triggering applications for distributed action execution and use of mixed media recognition as a control input |
US20060291396A1 (en) * | 2005-06-27 | 2006-12-28 | Monplaisir Hamilton | Optimizing driving directions |
US7626595B2 (en) * | 2005-08-01 | 2009-12-01 | Microsoft Corporation | Resolution independent image resource |
US7769772B2 (en) | 2005-08-23 | 2010-08-03 | Ricoh Co., Ltd. | Mixed media reality brokerage network with layout-independent recognition |
US8818076B2 (en) | 2005-09-01 | 2014-08-26 | Victor Shenkar | System and method for cost-effective, high-fidelity 3D-modeling of large-scale urban environments |
US7698061B2 (en) | 2005-09-23 | 2010-04-13 | Scenera Technologies, Llc | System and method for selecting and presenting a route to a user |
US8731585B2 (en) | 2006-02-10 | 2014-05-20 | Telecommunications Systems, Inc. | Intelligent reverse geocoding |
US7899468B2 (en) | 2005-09-30 | 2011-03-01 | Telecommunication Systems, Inc. | Location sensitive messaging |
US20070078596A1 (en) | 2005-09-30 | 2007-04-05 | John Grace | Landmark enhanced directions |
KR101126506B1 (en) * | 2005-10-28 | 2012-03-29 | 주식회사 현대오토넷 | Smart navigation system using portable memory and control method thereof |
KR100641791B1 (en) * | 2006-02-14 | 2006-11-02 | (주)올라웍스 | Tagging Method and System for Digital Data |
WO2007102099A1 (en) * | 2006-03-07 | 2007-09-13 | Koninklijke Philips Electronics N.V. | Navigation service using pre-fetching |
US8315905B1 (en) * | 2006-03-30 | 2012-11-20 | Navadi, Inc. | Location based advertising systems |
US20070233375A1 (en) * | 2006-03-31 | 2007-10-04 | Ashutosh Garg | Providing advertising in aerial imagery |
US7702456B2 (en) * | 2006-04-14 | 2010-04-20 | Scenera Technologies, Llc | System and method for presenting a computed route |
US20070299607A1 (en) * | 2006-06-27 | 2007-12-27 | Verizon Laboratories Inc. | Driving directions with landmark data |
KR100866206B1 (en) * | 2006-07-20 | 2008-10-30 | 삼성전자주식회사 | Apparatus and method for providing customized path guardence using a navigation game |
US8201076B2 (en) | 2006-07-31 | 2012-06-12 | Ricoh Co., Ltd. | Capturing symbolic information from documents upon printing |
US9176984B2 (en) | 2006-07-31 | 2015-11-03 | Ricoh Co., Ltd | Mixed media reality retrieval of differentially-weighted links |
US9020966B2 (en) | 2006-07-31 | 2015-04-28 | Ricoh Co., Ltd. | Client device for interacting with a mixed media reality recognition system |
US8073263B2 (en) | 2006-07-31 | 2011-12-06 | Ricoh Co., Ltd. | Multi-classifier selection and monitoring for MMR-based image recognition |
US9063952B2 (en) | 2006-07-31 | 2015-06-23 | Ricoh Co., Ltd. | Mixed media reality recognition with image tracking |
US8676810B2 (en) | 2006-07-31 | 2014-03-18 | Ricoh Co., Ltd. | Multiple index mixed media reality recognition using unequal priority indexes |
US8489987B2 (en) | 2006-07-31 | 2013-07-16 | Ricoh Co., Ltd. | Monitoring and analyzing creation and usage of visual content using image and hotspot interaction |
US8874145B2 (en) * | 2006-08-02 | 2014-10-28 | Telecommunication Systems, Inc. | Personal location code broker |
US7957751B2 (en) | 2006-08-02 | 2011-06-07 | Telecommunication Systems, Inc. | Personal location code |
US9031777B2 (en) * | 2006-08-08 | 2015-05-12 | Verizon Patent And Licensing Inc. | Driving directions with selective printing |
US20080147319A1 (en) * | 2006-12-19 | 2008-06-19 | Verizon Laboratories Inc. | Landmark selection tools for map systems and methods |
US20080147366A1 (en) * | 2006-12-19 | 2008-06-19 | Science Application International Corporation | System and method for displaying simulation data and visualization data |
US8744758B2 (en) * | 2006-12-19 | 2014-06-03 | Verizon Patent And Licensing Inc. | Driving directions printed text scaling |
US7768395B2 (en) | 2007-01-19 | 2010-08-03 | Gold Steven K | Brand mapping |
EP2163169B1 (en) * | 2007-03-20 | 2012-12-26 | Miche Bag, LLC | Systems and methods for customizing handbags |
US9222797B2 (en) | 2007-04-17 | 2015-12-29 | Esther Abramovich Ettinger | Device, system and method of contact-based routing and guidance |
US9243920B2 (en) | 2007-04-17 | 2016-01-26 | Esther Abramovich Ettinger | System and method for adapting the routing information provided by a mapping or routing device |
US8930135B2 (en) | 2007-04-17 | 2015-01-06 | Esther Abramovich Ettinger | Device, system and method of landmark-based routing and guidance |
KR100875375B1 (en) * | 2007-05-28 | 2008-12-23 | 팅크웨어(주) | Intersection guidance method and navigation system using point of interest information |
US7912637B2 (en) * | 2007-06-25 | 2011-03-22 | Microsoft Corporation | Landmark-based routing |
US20080319658A1 (en) * | 2007-06-25 | 2008-12-25 | Microsoft Corporation | Landmark-based routing |
US8065078B2 (en) * | 2007-08-10 | 2011-11-22 | Microsoft Corporation | Conveying locations in spoken dialog systems |
EP2031352A1 (en) * | 2007-08-29 | 2009-03-04 | Wayfinder Systems AB | Pre-fetching navigation maps |
US8374782B2 (en) | 2007-08-29 | 2013-02-12 | Vodafone Group Services Limited | Pre-fetching navigation maps |
US9464910B2 (en) * | 2007-08-31 | 2016-10-11 | Navigation Solutions, Llc | Navigation system with web interface |
US20090143125A1 (en) * | 2007-11-29 | 2009-06-04 | Microsoft Corporation | Population of landmarks for use with a map application |
US8155877B2 (en) | 2007-11-29 | 2012-04-10 | Microsoft Corporation | Location-to-landmark |
TWI352192B (en) * | 2007-12-31 | 2011-11-11 | Htc Corp | Satellite navigation method and system |
JP2011515717A (en) * | 2008-03-24 | 2011-05-19 | グーグル インコーポレイテッド | Panoramic image in driving instructions |
US8515674B2 (en) * | 2008-05-22 | 2013-08-20 | Mapquest, Inc. | Systems and methods for collecting and using user-contributed map data |
DE102008024920A1 (en) * | 2008-05-23 | 2009-12-10 | Egger, Kurt | Digital vehicle travel recording device, has exchangeable data medium e.g. secure digital card and universal serial bus stick, for recording data stored in ring memory by key press or sensor actuation |
US9200901B2 (en) * | 2008-06-19 | 2015-12-01 | Microsoft Technology Licensing, Llc | Predictive services for devices supporting dynamic direction information |
US20100009662A1 (en) | 2008-06-20 | 2010-01-14 | Microsoft Corporation | Delaying interaction with points of interest discovered based on directional device information |
US20090319166A1 (en) * | 2008-06-20 | 2009-12-24 | Microsoft Corporation | Mobile computing services based on devices with dynamic direction information |
US20100125406A1 (en) * | 2008-11-19 | 2010-05-20 | Nokia Corporation | Methods, apparatuses, and computer program products for providing point of interest navigation services |
US9062982B2 (en) * | 2008-12-15 | 2015-06-23 | Blackberry Limited | Pre-loading waypoint data |
US20100228612A1 (en) * | 2009-03-09 | 2010-09-09 | Microsoft Corporation | Device transaction model and services based on directional information of device |
US20120047087A1 (en) | 2009-03-25 | 2012-02-23 | Waldeck Technology Llc | Smart encounters |
US8176045B2 (en) | 2009-03-27 | 2012-05-08 | Google Inc. | Systems and methods for cross-street identification |
US8060302B2 (en) * | 2009-03-31 | 2011-11-15 | Microsoft Corporation | Visual assessment of landmarks |
US8385660B2 (en) | 2009-06-24 | 2013-02-26 | Ricoh Co., Ltd. | Mixed media reality indexing and retrieval for repeated content |
US8872767B2 (en) | 2009-07-07 | 2014-10-28 | Microsoft Corporation | System and method for converting gestures into digital graffiti |
US8442767B2 (en) * | 2009-08-28 | 2013-05-14 | Navteq B.V. | Method of operating a navigation system to provide route guidance |
US20110098910A1 (en) * | 2009-10-22 | 2011-04-28 | Nokia Corporation | Method and apparatus for intelligent guidance using markers |
US8694383B2 (en) | 2009-12-09 | 2014-04-08 | Microsoft Corporation | Path queries |
CN102753940A (en) * | 2009-12-17 | 2012-10-24 | 通腾科技股份有限公司 | Navigation apparatus, audible instruction generation system and method of generating audible instructions |
US8234063B2 (en) * | 2009-12-18 | 2012-07-31 | Telenav, Inc. | Navigation system with location profiling and method of operation thereof |
JP5696996B2 (en) | 2009-12-28 | 2015-04-08 | サイバーアイ・エンタテインメント株式会社 | Image display system |
EP2553396B1 (en) | 2010-03-26 | 2018-12-19 | Koninklijke Philips N.V. | A method and navigation device for providing at least one route |
US8988468B2 (en) | 2011-01-21 | 2015-03-24 | Wishabi Inc. | Interactive flyer system |
JP5625987B2 (en) * | 2011-02-16 | 2014-11-19 | アイシン・エィ・ダブリュ株式会社 | Guide device, guide method, and guide program |
US20150117796A1 (en) * | 2011-03-28 | 2015-04-30 | Google Inc. | Method and system for prioritizing points of interest for display in a map |
US8874366B2 (en) | 2011-05-23 | 2014-10-28 | Microsoft Corporation | First waypoint distance |
US9058331B2 (en) | 2011-07-27 | 2015-06-16 | Ricoh Co., Ltd. | Generating a conversation in a social network based on visual search results |
US8683008B1 (en) | 2011-08-04 | 2014-03-25 | Google Inc. | Management of pre-fetched mapping data incorporating user-specified locations |
US9505497B2 (en) | 2011-08-10 | 2016-11-29 | Honeywell International Inc. | Systems and methods for a virtual terrain display |
US8938257B2 (en) * | 2011-08-19 | 2015-01-20 | Qualcomm, Incorporated | Logo detection for indoor positioning |
US8204966B1 (en) | 2011-09-26 | 2012-06-19 | Google Inc. | Map tile data pre-fetching based on user activity analysis |
US8280414B1 (en) | 2011-09-26 | 2012-10-02 | Google Inc. | Map tile data pre-fetching based on mobile device generated event analysis |
US20130103608A1 (en) * | 2011-10-21 | 2013-04-25 | Ebay Inc. | Location Determination and Map Building |
US9275374B1 (en) | 2011-11-15 | 2016-03-01 | Google Inc. | Method and apparatus for pre-fetching place page data based upon analysis of user activities |
US8886715B1 (en) | 2011-11-16 | 2014-11-11 | Google Inc. | Dynamically determining a tile budget when pre-fetching data in a client device |
US9063951B1 (en) | 2011-11-16 | 2015-06-23 | Google Inc. | Pre-fetching map data based on a tile budget |
US8711181B1 (en) | 2011-11-16 | 2014-04-29 | Google Inc. | Pre-fetching map data using variable map tile radius |
US9305107B2 (en) | 2011-12-08 | 2016-04-05 | Google Inc. | Method and apparatus for pre-fetching place page data for subsequent display on a mobile computing device |
US9197713B2 (en) | 2011-12-09 | 2015-11-24 | Google Inc. | Method and apparatus for pre-fetching remote resources for subsequent display on a mobile computing device |
US8803920B2 (en) | 2011-12-12 | 2014-08-12 | Google Inc. | Pre-fetching map tile data along a route |
US9389088B2 (en) | 2011-12-12 | 2016-07-12 | Google Inc. | Method of pre-fetching map data for rendering and offline routing |
US8688379B1 (en) * | 2012-01-10 | 2014-04-01 | Google Inc. | Method and system for generating drive time isocontours using nested graphs |
AU2013214642A1 (en) | 2012-02-02 | 2014-09-18 | Flipp Corporation | Dynamic digital flyer system |
US9494427B2 (en) | 2012-04-25 | 2016-11-15 | Tyrell Gray | System and method for providing a directional interface |
US9332387B2 (en) | 2012-05-02 | 2016-05-03 | Google Inc. | Prefetching and caching map data based on mobile network coverage |
US8694246B2 (en) | 2012-05-15 | 2014-04-08 | Qualcomm Incorporated | Methods and systems for displaying enhanced turn-by-turn guidance on a personal navigation device |
US8849942B1 (en) | 2012-07-31 | 2014-09-30 | Google Inc. | Application programming interface for prefetching map data |
US20140039789A1 (en) * | 2012-08-06 | 2014-02-06 | International Business Machines Corporation | Dynamic progressive map granularity for navigation |
US8898016B2 (en) | 2013-03-15 | 2014-11-25 | Applied Minds, Llc | Method and apparatus for two-stage planning |
US11022446B2 (en) | 2013-03-15 | 2021-06-01 | Applied Invention, Llc | Method and apparatus for two-stage planning |
EP2799817A3 (en) * | 2013-04-30 | 2015-09-09 | GN Store Nord A/S | An apparatus and a method of providing information in relation to a point of interest to a user |
US11022456B2 (en) * | 2013-07-25 | 2021-06-01 | Nokia Technologies Oy | Method of audio processing and audio processing apparatus |
EP3077985B1 (en) * | 2013-12-04 | 2020-05-27 | Urthecast Corp. | Systems and methods for processing distributing earth observation images |
EP2927637A1 (en) * | 2014-04-01 | 2015-10-07 | Nokia Technologies OY | Association between a point of interest and an obejct |
WO2015187124A1 (en) * | 2014-06-02 | 2015-12-10 | Hewlett-Packard Development Company, L.P. | Waypoint navigator |
WO2015192056A1 (en) | 2014-06-13 | 2015-12-17 | Urthecast Corp. | Systems and methods for processing and providing terrestrial and/or space-based earth observation video |
CA2980920C (en) | 2015-03-25 | 2023-09-26 | King Abdulaziz City Of Science And Technology | Apparatus and methods for synthetic aperture radar with digital beamforming |
US9528848B2 (en) * | 2015-03-30 | 2016-12-27 | Alpine Electronics, Inc. | Method of displaying point on navigation map |
US10615513B2 (en) | 2015-06-16 | 2020-04-07 | Urthecast Corp | Efficient planar phased array antenna assembly |
MX2018001329A (en) | 2015-08-07 | 2018-04-30 | Wal Mart Stores Inc | Apparatus and method for updating directions to a container. |
EP3380864A4 (en) | 2015-11-25 | 2019-07-03 | Urthecast Corp. | Synthetic aperture radar imaging apparatus and methods |
KR102565485B1 (en) * | 2016-01-11 | 2023-08-14 | 한국전자통신연구원 | Server and Method for Providing City Street Search Service |
US10054460B2 (en) | 2016-03-28 | 2018-08-21 | Microsoft Technology Licensing, Llc | Processing map-related user input to detect route requests |
DE102016205867A1 (en) * | 2016-04-08 | 2017-10-12 | Robert Bosch Gmbh | Method for determining a pose of an at least partially automated vehicle using different types of landmarks |
US10126141B2 (en) * | 2016-05-02 | 2018-11-13 | Google Llc | Systems and methods for using real-time imagery in navigation |
CN106293357B (en) * | 2016-08-02 | 2019-08-02 | 网易(杭州)网络有限公司 | The display methods and device of map label picture |
US10168173B2 (en) | 2016-10-26 | 2019-01-01 | Google Llc | Systems and methods for using visual landmarks in initial navigation |
US10692023B2 (en) * | 2017-05-12 | 2020-06-23 | International Business Machines Corporation | Personal travel assistance system and method for traveling through a transport hub |
US10346773B2 (en) | 2017-05-12 | 2019-07-09 | International Business Machines Corporation | Coordinating and providing navigation for a group of people traveling together in a transport hub |
CA3064735C (en) | 2017-05-23 | 2022-06-21 | Urthecast Corp. | Synthetic aperture radar imaging apparatus and methods |
CA3064586A1 (en) | 2017-05-23 | 2018-11-29 | King Abdullah City Of Science And Technology | Synthetic aperture radar imaging apparatus and methods for moving targets |
CA3083033A1 (en) | 2017-11-22 | 2019-11-28 | Urthecast Corp. | Synthetic aperture radar apparatus and methods |
US10699571B2 (en) * | 2017-12-04 | 2020-06-30 | Ford Global Technologies, Llc | High definition 3D mapping |
US10788332B2 (en) | 2017-12-07 | 2020-09-29 | International Business Machines Corporation | Route navigation based on user feedback |
JP7065656B2 (en) * | 2018-03-19 | 2022-05-12 | 本田技研工業株式会社 | Information presentation system, information presentation method, and program |
US11195102B2 (en) | 2018-04-23 | 2021-12-07 | International Business Machines Corporation | Navigation and cognitive dialog assistance |
US11835354B2 (en) | 2018-09-06 | 2023-12-05 | Google Llc | Navigation directions with a familiar location as an intermediate destination |
JP7209746B2 (en) | 2018-09-06 | 2023-01-20 | グーグル エルエルシー | Displaying personalized landmarks in mapping applications |
WO2020086051A1 (en) * | 2018-10-22 | 2020-04-30 | Google Llc | Finding locally prominent semantic features for navigation and geocoding |
US11107099B2 (en) * | 2018-12-21 | 2021-08-31 | Google Llc | Brand penetration determination system using image semantic content |
EP3791350A1 (en) | 2018-12-21 | 2021-03-17 | Google LLC | Brand penetration determination system using image semantic content |
EP3822585A1 (en) * | 2019-11-15 | 2021-05-19 | Volkswagen AG | Vehicle navigation system and method for providing turn guidance for a driver of a vehicle |
CN111238519B (en) * | 2020-01-06 | 2022-05-03 | 华侨大学 | Multi-unmanned vehicle road finding method based on topological map and conflict elimination strategy |
JP2022178701A (en) * | 2021-05-20 | 2022-12-02 | フォルシアクラリオン・エレクトロニクス株式会社 | navigation device |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060200383A1 (en) * | 2005-03-03 | 2006-09-07 | Arutunian Ethan B | Enhanced maps, such as for lead generation |
US20060241859A1 (en) * | 2005-04-21 | 2006-10-26 | Microsoft Corporation | Virtual earth real-time advertising |
US20060271286A1 (en) * | 2005-05-27 | 2006-11-30 | Outland Research, Llc | Image-enhanced vehicle navigation systems and methods |
US20070154067A1 (en) * | 1998-10-23 | 2007-07-05 | Facet Technology Corp. | Method and apparatus for identifying objects depicted in a videostream |
US20070198182A1 (en) * | 2004-09-30 | 2007-08-23 | Mona Singh | Method for incorporating images with a user perspective in navigation |
Family Cites Families (120)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4972319A (en) | 1987-09-25 | 1990-11-20 | Delorme David M | Electronic global map generating system |
US5051928A (en) | 1987-12-28 | 1991-09-24 | Dubner Computer Systems, Inc. | Color correction for video graphics system |
JP2609744B2 (en) | 1989-07-14 | 1997-05-14 | 株式会社日立製作所 | Image display method and image display device |
JPH06287461A (en) * | 1993-04-07 | 1994-10-11 | Toyota Central Res & Dev Lab Inc | Dichroic crystal |
US6002809A (en) | 1993-04-15 | 1999-12-14 | International Business Machines Corporation | Digital image processor for image scaling |
US5625668A (en) | 1994-04-12 | 1997-04-29 | Trimble Navigation Limited | Position reporting cellular telephone |
US5848373A (en) | 1994-06-24 | 1998-12-08 | Delorme Publishing Company | Computer aided map location system |
US6321158B1 (en) | 1994-06-24 | 2001-11-20 | Delorme Publishing Company | Integrated routing/mapping information |
US5802492A (en) | 1994-06-24 | 1998-09-01 | Delorme Publishing Company, Inc. | Computer aided routing and positioning system |
US5559707A (en) | 1994-06-24 | 1996-09-24 | Delorme Publishing Company | Computer aided routing system |
US5613051A (en) | 1994-12-21 | 1997-03-18 | Harris Corp. | Remote image exploitation display system and method |
CA2168641C (en) | 1995-02-03 | 2000-03-28 | Tetsuya Kitamura | Image information encoding/decoding system |
IL117053A0 (en) | 1995-05-01 | 1996-06-18 | Motorola Inc | A display method for data images |
US5646751A (en) | 1995-05-25 | 1997-07-08 | Xerox Corporation | Encoding method for deferred anti-aliasing |
US5870097A (en) | 1995-08-04 | 1999-02-09 | Microsoft Corporation | Method and system for improving shadowing in a graphics rendering system |
US5737533A (en) | 1995-10-26 | 1998-04-07 | Wegener Internet Projects Bv | System for generating a virtual reality scene in response to a database search |
US5760783A (en) | 1995-11-06 | 1998-06-02 | Silicon Graphics, Inc. | Method and system for providing texture using a selected portion of a texture map |
DE19549306A1 (en) | 1995-12-22 | 1997-07-03 | Art & Com Medientechnologie Un | Method and device for the visual representation of spatial data |
US5910805A (en) | 1996-01-11 | 1999-06-08 | Oclc Online Computer Library Center | Method for displaying bitmap derived text at a display having limited pixel-to-pixel spacing resolution |
US6098015A (en) | 1996-04-23 | 2000-08-01 | Aisin Aw Co., Ltd. | Navigation system for vehicles and storage medium |
US6054990A (en) | 1996-07-05 | 2000-04-25 | Tran; Bao Q. | Computer system with handwriting annotation |
US5842009A (en) | 1996-09-30 | 1998-11-24 | Apple Computer, Inc. | System for automatically retrieving information relevant to a user's scheduled event |
US5938709A (en) | 1996-11-22 | 1999-08-17 | Case Corporation | Panning display of GPS field maps |
JP3644473B2 (en) | 1997-08-07 | 2005-04-27 | アイシン・エィ・ダブリュ株式会社 | Map display device and recording medium |
US6496189B1 (en) | 1997-09-29 | 2002-12-17 | Skyline Software Systems Ltd. | Remote landscape display and pilot training |
US6111583A (en) | 1997-09-29 | 2000-08-29 | Skyline Software Systems Ltd. | Apparatus and method for three-dimensional terrain rendering |
US6522336B1 (en) | 1997-10-31 | 2003-02-18 | Hewlett-Packard Company | Three-dimensional graphics rendering apparatus and method |
US6199014B1 (en) | 1997-12-23 | 2001-03-06 | Walker Digital, Llc | System for providing driving directions with visual cues |
US6266614B1 (en) | 1997-12-24 | 2001-07-24 | Wendell Alumbaugh | Travel guide |
US6144338A (en) | 1998-03-17 | 2000-11-07 | Prc Public Sector. Inc. | Predictive drop and load algorithm for an object-based geographical information system |
US6262741B1 (en) | 1998-03-17 | 2001-07-17 | Prc Public Sector, Inc. | Tiling of object-based geographic information system (GIS) |
US6247019B1 (en) | 1998-03-17 | 2001-06-12 | Prc Public Sector, Inc. | Object-based geographic information system (GIS) |
US6192314B1 (en) | 1998-03-25 | 2001-02-20 | Navigation Technologies Corp. | Method and system for route calculation in a navigation application |
US6167382A (en) | 1998-06-01 | 2000-12-26 | F.A.C. Services Group, L.P. | Design and production of print advertising and commercial display materials over the Internet |
EP0967556A2 (en) | 1998-06-26 | 1999-12-29 | Hewlett-Packard Company | Flat image delivery server |
US6573905B1 (en) | 1999-11-09 | 2003-06-03 | Broadcom Corporation | Video and graphics system with parallel processing of graphics windows |
US6523061B1 (en) | 1999-01-05 | 2003-02-18 | Sri International, Inc. | System, method, and article of manufacture for agent-based navigation in a speech-based data navigation system |
US6208934B1 (en) | 1999-01-19 | 2001-03-27 | Navigation Technologies Corp. | Method and system for providing walking instructions with route guidance in a navigation program |
US6182010B1 (en) | 1999-01-28 | 2001-01-30 | International Business Machines Corporation | Method and apparatus for displaying real-time visual information on an automobile pervasive computing client |
US6346938B1 (en) | 1999-04-27 | 2002-02-12 | Harris Corporation | Computer-resident mechanism for manipulating, navigating through and mensurating displayed image of three-dimensional geometric model |
US6760783B1 (en) * | 1999-05-21 | 2004-07-06 | Intel Corporation | Virtual interrupt mechanism |
US6421053B1 (en) | 1999-05-24 | 2002-07-16 | International Business Machines Corporation | Block rendering method for a graphics subsystem |
US6307573B1 (en) | 1999-07-22 | 2001-10-23 | Barbara L. Barros | Graphic-information flow method and system for visually analyzing patterns and relationships |
WO2001027822A1 (en) | 1999-10-14 | 2001-04-19 | Yamaha Hatsudoki Kabushiki Kaisha | Trip planning system |
US6538656B1 (en) | 1999-11-09 | 2003-03-25 | Broadcom Corporation | Video and graphics system with a data transport processor |
US6826472B1 (en) | 1999-12-10 | 2004-11-30 | Tele Atlas North America, Inc. | Method and apparatus to generate driving guides |
DE19963766A1 (en) | 1999-12-30 | 2001-07-05 | Bosch Gmbh Robert | Method for operating a navigation system |
AU4148901A (en) | 2000-02-14 | 2001-08-27 | Virtuacities, Inc. | Methods and systems for presenting a virtual representation of a real city |
US7136915B2 (en) | 2000-03-14 | 2006-11-14 | Rieger Iii Charles J | System for communicating through maps |
WO2001071484A1 (en) | 2000-03-17 | 2001-09-27 | Vicinity Corp. | System and method for non-uniform scaled mapping |
US6430553B1 (en) | 2000-03-22 | 2002-08-06 | Exactone.Com, Inc. | Method and apparatus for parsing data |
AU2001251123A1 (en) | 2000-03-30 | 2001-10-15 | Iqbal A. Talib | Methods and systems for enabling efficient retrieval of data from data collections |
US7082605B2 (en) | 2000-03-31 | 2006-07-25 | Vidus Limited | Contingency planning in a scheduling process |
CA2403763A1 (en) | 2000-03-31 | 2001-10-11 | British Telecommunications Public Limited Company | Handling unscheduled tasks in a scheduling process |
NZ504934A (en) | 2000-06-02 | 2003-08-29 | Compudigm Int Ltd | Travel route planning optimized in accordance with traveller constraints |
US6718258B1 (en) | 2000-06-08 | 2004-04-06 | Navigation Technologies Corp | Method and system for obtaining user feedback regarding geographic data |
US20020095328A1 (en) | 2000-06-14 | 2002-07-18 | Garret Swart | Method and apparatus for profiling clients of a database-driven transaction system and marketing according to profile data |
WO2002003234A2 (en) | 2000-06-30 | 2002-01-10 | Troy Schultz | Method and apparatus for a gis based search engine utilizing real time advertising |
US6766320B1 (en) | 2000-08-24 | 2004-07-20 | Microsoft Corporation | Search engine with natural language-based robust parsing for user query and relevance feedback learning |
US7174301B2 (en) | 2000-10-23 | 2007-02-06 | Costar Group, Inc. | System and method for accessing geographic-based data |
US7640204B2 (en) | 2000-10-23 | 2009-12-29 | Costar Group, Inc. | System and method for collection, distribution, and use of information in connection with commercial real estate |
US6724382B2 (en) | 2000-12-04 | 2004-04-20 | Wildtangent, Inc. | Method and apparatus for distributing and displaying maps electronically |
US20020067379A1 (en) | 2000-12-04 | 2002-06-06 | Kenyon Jeremy A. | Method and apparatus for distributing and displaying maps electronically |
US20020067374A1 (en) | 2000-12-04 | 2002-06-06 | Kenyon Jeremy A. | Method and apparatus for distributing and displaying maps electronically |
US20020163654A1 (en) | 2001-03-23 | 2002-11-07 | Oak Technology, Inc. | System and method of simulated increase in image resolution |
US20020159657A1 (en) | 2001-04-27 | 2002-10-31 | Delorme Publishing Company | Folding holder for maps and related travel information printouts |
WO2002088979A1 (en) | 2001-05-02 | 2002-11-07 | Bitstream Inc. | Methods and systems for displaying media in scaled manner and/or in orientation |
US20040153434A1 (en) | 2001-05-08 | 2004-08-05 | Cogen Jeffrey Morris | Method and apparatus for documenting use of a trademark or service mark |
US7054741B2 (en) | 2002-02-11 | 2006-05-30 | Landnet Corporation | Land software tool |
US6629034B1 (en) | 2001-06-06 | 2003-09-30 | Navigation Technologies Corp. | Driving profile method and system |
US6532418B2 (en) | 2001-07-02 | 2003-03-11 | Samsung Electronics, Co., Ltd. | Vehicle navigation network, apparatus and method for use in a mobile telecommunication system |
JP2003067721A (en) | 2001-08-24 | 2003-03-07 | Pioneer Electronic Corp | Map image display system and method |
CN1507553A (en) | 2001-08-31 | 2004-06-23 | ����Aw��ʽ���� | Information display system |
US7389181B2 (en) | 2004-08-31 | 2008-06-17 | Visre, Inc. | Apparatus and method for producing video drive-by data corresponding to a geographic location |
US6584401B2 (en) | 2001-11-27 | 2003-06-24 | Hewlett-Packard Development Company, Lp. | Automatic gathering and analysis of data on commute paths |
US6741926B1 (en) | 2001-12-06 | 2004-05-25 | Bellsouth Intellectual Property Corporation | Method and system for reporting automotive traffic conditions in response to user-specific requests |
US7107285B2 (en) | 2002-03-16 | 2006-09-12 | Questerra Corporation | Method, system, and program for an improved enterprise spatial system |
US9953590B2 (en) | 2002-04-11 | 2018-04-24 | Samsung Display Co., Ltd. | Color display devices and methods with enhanced attributes |
US6891540B2 (en) | 2002-10-17 | 2005-05-10 | Silicon Integrated Systems Corp. | Apparatus and method for line drawing |
JP4005904B2 (en) | 2002-11-27 | 2007-11-14 | 松下電器産業株式会社 | Display device and display method |
US7054745B1 (en) | 2003-09-03 | 2006-05-30 | Microsoft Corporation | Method and system for generating driving directions |
US7359873B2 (en) | 2003-09-19 | 2008-04-15 | Yahoo, Inc. | Self-service catalog manager for stores implemented on a communications network |
US7081902B1 (en) | 2003-09-24 | 2006-07-25 | Nvidia Corporation | Apparatus, system, and method for gamma correction of smoothed primitives |
US7050068B1 (en) | 2003-12-02 | 2006-05-23 | Nvidia Corporation | Generation of jittered sub-pixel samples using programmable sub-pixel offsets |
US8452526B2 (en) | 2003-12-15 | 2013-05-28 | Gary Ignatin | Estimation of roadway travel information based on historical travel data |
US20050171686A1 (en) | 2004-01-30 | 2005-08-04 | Davis Scott B. | Method and apparatus for obtaining and providing information related to a point-of-interest |
US7610219B2 (en) | 2004-02-17 | 2009-10-27 | Omar Farooq Sayed | System and methods for assembly of a web site for an online store by a seller |
US10417298B2 (en) | 2004-12-02 | 2019-09-17 | Insignio Technologies, Inc. | Personalized content processing and delivery system and media |
US7680694B2 (en) | 2004-03-11 | 2010-03-16 | American Express Travel Related Services Company, Inc. | Method and apparatus for a user to shop online in a three dimensional virtual reality setting |
CA2820249C (en) | 2004-03-23 | 2016-07-19 | Google Inc. | A digital mapping system |
US7599790B2 (en) | 2004-03-23 | 2009-10-06 | Google Inc. | Generating and serving tiles in a digital mapping system |
US20050216287A1 (en) | 2004-03-29 | 2005-09-29 | Crosby Michael W | Method for integrating an entrepreneur's web site and a store front web site |
US7373244B2 (en) | 2004-04-20 | 2008-05-13 | Keith Kreft | Information mapping approaches |
US7644239B2 (en) | 2004-05-03 | 2010-01-05 | Microsoft Corporation | Non-volatile memory cache performance improvement |
US7558595B2 (en) | 2004-06-25 | 2009-07-07 | Sony Ericsson Mobile Communications Ab | Mobile terminals, methods, and program products that generate communication information based on characters recognized in image data |
US20060017562A1 (en) | 2004-07-20 | 2006-01-26 | Bachelder Aaron D | Distributed, roadside-based real-time ID recognition system and method |
US20060041375A1 (en) | 2004-08-19 | 2006-02-23 | Geographic Data Technology, Inc. | Automated georeferencing of digitized map images |
US8150617B2 (en) | 2004-10-25 | 2012-04-03 | A9.Com, Inc. | System and method for displaying location-specific images on a mobile device |
US7831384B2 (en) | 2004-10-29 | 2010-11-09 | Aol Inc. | Determining a route to destination based on partially completed route |
US7835859B2 (en) | 2004-10-29 | 2010-11-16 | Aol Inc. | Determining a route to a destination based on partially completed route |
JP4156584B2 (en) | 2004-11-30 | 2008-09-24 | 株式会社ケンウッド | Navigation device, method and program |
US7260474B1 (en) | 2005-02-08 | 2007-08-21 | Cellco Partnership | GIS point-to-point routing for directions |
US20060206363A1 (en) | 2005-03-13 | 2006-09-14 | Gove Jeremy J | Group travel planning, optimization, synchronization and coordination software tool and processes for travel arrangements for transportation and lodging for multiple people from multiple geographic locations, domestic and global, to a single destination or series of destinations |
US7353034B2 (en) | 2005-04-04 | 2008-04-01 | X One, Inc. | Location sharing and tracking using mobile phones or other wireless devices |
US7617296B2 (en) | 2005-04-22 | 2009-11-10 | Realnetworks, Inc. | Data compilation system and method |
WO2006121986A2 (en) | 2005-05-06 | 2006-11-16 | Facet Technology Corp. | Network-based navigation system having virtual drive-thru advertisements integrated with actual imagery from along a physical route |
US20070022007A1 (en) | 2005-06-14 | 2007-01-25 | Mystorecredit.Com | System and method for a customer loyalty reward system utilizing a shopping search portal, a payment transfer agent and email marketing |
US20070011171A1 (en) | 2005-07-08 | 2007-01-11 | Nurminen Jukka K | System and method for operation control functionality |
EP1934910A4 (en) | 2005-08-26 | 2011-03-16 | Spot Runner Inc | Systems and methods for media planning, ad production, ad placement and content customization |
US20080052150A1 (en) | 2005-08-26 | 2008-02-28 | Spot Runner, Inc., A Delaware Corporation | Systems and Methods For Media Planning, Ad Production, and Ad Placement For Radio |
US8700586B2 (en) | 2005-10-31 | 2014-04-15 | Yahoo! Inc. | Clickable map interface |
US8595633B2 (en) | 2005-10-31 | 2013-11-26 | Yahoo! Inc. | Method and system for displaying contextual rotating advertisements |
US20070100801A1 (en) | 2005-10-31 | 2007-05-03 | Celik Aytek E | System for selecting categories in accordance with advertising |
US20070106468A1 (en) | 2005-11-07 | 2007-05-10 | France Telecom | Product, service and activity based interactive trip mapping system, method, and computer program product |
EP1952371A2 (en) | 2005-11-09 | 2008-08-06 | Micro Target Media Holdings Inc. | Advertising display for a portable structure |
US7640104B2 (en) | 2006-02-27 | 2009-12-29 | Xanavi Informatics Corporation | Vehicle navigation system and method for displaying waypoint information |
US8224695B2 (en) | 2006-03-31 | 2012-07-17 | Google Inc. | Monetizing service calls through advertising |
US7693652B2 (en) | 2006-04-05 | 2010-04-06 | Microsoft Corporation | Waypoint adjustment and advertisement for flexible routing |
RU2008149112A (en) | 2006-06-30 | 2010-06-20 | Теле Атлас Норт Америка, Инк. (Us) | METHOD AND SYSTEM FOR COLLECTING USER REQUESTS FOR UPDATING REGARDING GEOGRAPHIC DATA TO SUPPORT AUTOMATED ANALYSIS, PROCESSING AND UPDATES OF GEOGRAPHIC DATA |
US8666821B2 (en) | 2006-08-28 | 2014-03-04 | Microsoft Corporation | Selecting advertisements based on serving area and map area |
-
2005
- 2005-07-13 US US11/181,386 patent/US7831387B2/en active Active
-
2006
- 2006-07-10 WO PCT/US2006/026745 patent/WO2007008809A2/en active Application Filing
- 2006-08-22 US US11/466,326 patent/US7920968B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070154067A1 (en) * | 1998-10-23 | 2007-07-05 | Facet Technology Corp. | Method and apparatus for identifying objects depicted in a videostream |
US20070198182A1 (en) * | 2004-09-30 | 2007-08-23 | Mona Singh | Method for incorporating images with a user perspective in navigation |
US20060200383A1 (en) * | 2005-03-03 | 2006-09-07 | Arutunian Ethan B | Enhanced maps, such as for lead generation |
US20060241859A1 (en) * | 2005-04-21 | 2006-10-26 | Microsoft Corporation | Virtual earth real-time advertising |
US20060271286A1 (en) * | 2005-05-27 | 2006-11-30 | Outland Research, Llc | Image-enhanced vehicle navigation systems and methods |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8219316B2 (en) | 2008-11-14 | 2012-07-10 | Google Inc. | System and method for storing and providing routes |
US8433513B2 (en) | 2008-11-14 | 2013-04-30 | Google Inc. | System and method for storing and providing routes |
US8762058B2 (en) | 2008-11-14 | 2014-06-24 | Google Inc. | System and method for storing and providing routes |
US9612125B2 (en) | 2008-11-14 | 2017-04-04 | Google Inc. | System and method for storing and providing routes |
US10247563B2 (en) | 2008-11-14 | 2019-04-02 | Google Llc | System and method for storing and providing routes |
US11243088B2 (en) | 2008-11-14 | 2022-02-08 | Google Llc | System and method for storing and providing routes |
US8489326B1 (en) | 2010-02-09 | 2013-07-16 | Google Inc. | Placemarked based navigation and ad auction based on placemarks |
US8754777B1 (en) | 2010-08-13 | 2014-06-17 | Google Inc. | System and method for predicting user route and destination |
Also Published As
Publication number | Publication date |
---|---|
US20070016368A1 (en) | 2007-01-18 |
US7920968B2 (en) | 2011-04-05 |
US7831387B2 (en) | 2010-11-09 |
WO2007008809A3 (en) | 2008-01-31 |
US20050288859A1 (en) | 2005-12-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7831387B2 (en) | Visually-oriented driving directions in digital mapping system | |
US20200408555A1 (en) | Electronic map display method and apparatus | |
EP1614997B1 (en) | Method of operating a navigation system using images | |
US8364398B2 (en) | Method of operating a navigation system to provide route guidance | |
JP5496997B2 (en) | Method of collecting geographic database information for use in a navigation system | |
EP0897170B1 (en) | Location floor number display device in navigation apparatus | |
US8751156B2 (en) | Method of operating a navigation system using images | |
US8290705B2 (en) | Mobile navigation system with graphic crime-risk display | |
US20020072848A1 (en) | Landmark update system and navigation device | |
CN102003965A (en) | Method of operating a navigation system to provide route guidance | |
CN102003964A (en) | Method of operating a navigation system to provide route guidance | |
US20190287398A1 (en) | Dynamic natural guidance | |
JP2007240198A (en) | Navigation apparatus | |
JP4622126B2 (en) | Navigation device, navigation method, and navigation program | |
EP3923247A1 (en) | Method, apparatus, and system for projecting augmented reality navigation cues on user-selected surfaces | |
CN110741227A (en) | Landmark assisted navigation | |
JP2007256207A (en) | Position registering device, route search device, position registration method, position registration program, and recording medium | |
CN102235868A (en) | Method of operating a navigation system to provide route guidance | |
CN115406453A (en) | Navigation method, navigation device and computer storage medium | |
JP5322080B2 (en) | Map information display device and map information display method | |
JP2019144556A (en) | Information processing apparatus, method, and program | |
JP2007263580A (en) | Route search device, route search method, route search program, and recording medium | |
JP6542085B2 (en) | INFORMATION PROCESSING APPARATUS, METHOD, AND PROGRAM | |
JP2019144173A (en) | Navigation device, and navigation program | |
JP2010096980A (en) | Device and method for setting question |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 06786785 Country of ref document: EP Kind code of ref document: A2 |