US7673288B1 - Bypassing execution of a software test using a file cache - Google Patents
Bypassing execution of a software test using a file cache Download PDFInfo
- Publication number
- US7673288B1 US7673288B1 US11/264,732 US26473205A US7673288B1 US 7673288 B1 US7673288 B1 US 7673288B1 US 26473205 A US26473205 A US 26473205A US 7673288 B1 US7673288 B1 US 7673288B1
- Authority
- US
- United States
- Prior art keywords
- generating
- file cache
- respective key
- final result
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related, expires
Links
- 238000000034 method Methods 0.000 claims abstract description 32
- 230000004044 response Effects 0.000 claims abstract description 20
- 238000013522 software testing Methods 0.000 claims abstract description 19
- 238000012360 testing method Methods 0.000 claims description 57
- 238000004088 simulation Methods 0.000 claims description 32
- 239000013598 vector Substances 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 2
- 238000013507 mapping Methods 0.000 claims 1
- 230000002194 synthesizing effect Effects 0.000 claims 1
- 230000008569 process Effects 0.000 description 19
- 230000007547 defect Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 238000013461 design Methods 0.000 description 6
- 238000012795 verification Methods 0.000 description 5
- 235000019800 disodium phosphate Nutrition 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000013100 final test Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000004549 pulsed laser deposition Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3696—Methods or tools to render software testable
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
Definitions
- the present invention generally relates to speeding up software testing by using a file cache.
- Test T consists of two main steps: first, running S to generate a design, and second, simulating the generated design. Test T succeeds if the simulation produces expected results. Simulations are usually quite slow, so it is useful to eliminate simulation testing if practical. Since the simulation results depend on nothing except for the design being simulated (assuming the same simulation stimuli), the results are guaranteed to be good if the design being simulated is known to be identical to the design for which simulation succeeded previously.
- the present invention may address one or more of the above issues.
- Various embodiments of the invention provide a processor-implemented method for software testing.
- Intermediate results are generated for one or more software tests, with each intermediate result corresponding to a software test.
- a respective key is generated from each intermediate result. Whether the respective key is stored in a file cache, which stores a plurality of files under a plurality of keys, is determined for each intermediate result.
- Generation of a final result for the corresponding software test of each intermediate result is bypassed in response to the respective key being present in the file cache.
- a final result for the corresponding software test of each intermediate result is generated from the intermediate result in response to the respective key not being present in the file cache.
- the respective key is stored in the file cache in response to generation of each final result.
- a generator generates intermediate results for at least one software test, with each intermediate result corresponding to a software test.
- a hash generator generates a respective key from each intermediate result.
- an executor bypasses generation of a final result of the corresponding software test responsive to the respective key being present in a file cache, which stores files under keys.
- the executor also generates a final result of the corresponding software test from each intermediate result responsive to the respective key not being present in the file cache, and stores the respective key in the file cache responsive to each final result.
- FIG. 1 is a flow diagram of a process for software testing in accordance with various embodiments of the invention.
- FIG. 2 is a block diagram of a system for execution of software testing in accordance with various embodiments of the invention
- FIG. 3 is a flow diagram of a process for simulating an electronic circuit in accordance with various embodiments of the invention.
- FIG. 4 is a block diagram of a system for software testing in accordance with various embodiments of the invention.
- FIG. 5 is a block diagram of a programmable logic device (PLD) that may implement an electronic circuit that is simulated in accordance with various embodiments of the invention.
- PLD programmable logic device
- a file cache is generally used to store and retrieve files, and is organized as a collection of key/value pairs. Each value is a file or a set of files. Each key is an object that identifies the corresponding file or files. Usually keys are strings or hash values.
- a “general-purpose” file cache is a file cache that is able to store arbitrary kinds and numbers of files.
- a software test is to skip executing a portion of the test. For example, a first version of the software system may be modified during debugging to create a second version of the software system that fixes a defect discovered by a failing software test in the test suite.
- Various embodiments of the invention may skip re-executing a portion of a software test that has completed successfully on the first version of the software system because the skipped portion almost certainly would complete successfully on the second version of the software system. For certain software systems, execution of all of the software tests in the suite may take days or weeks.
- efficiency of verification may be increased by abbreviating software tests that are likely to pass again, because a passing test may not help identify any remaining defects in the software system. Instead, the remaining defects in the software system may be more quickly discovered by re-executing failing software tests and also executing software tests that have not been previously executed.
- a previously passing software test could become a failing software test after fixing a defect because the fix may introduce another defect or the fix may perturb the behavior of the software system such that a latent defect is exposed by the previously passing test.
- verification of a software system is not generally completed until every software test in the suite is verified to complete successfully on a final version of the software system. It will be appreciated that the skipping of passing software tests may be periodically, intermittently, or occasionally suspended, such that previously passing tests are executed again. Despite the possibility that a previously passing test may become a failing test, the overall efficiency of the verification of a software system has been found to be improved by generally abbreviating previously passing software tests.
- the execution of a software test may be split into generation of an intermediate result and generation of a final result from the intermediate result.
- a single key may be generated from the intermediate result.
- the key may be a checksum or hash of the intermediate result.
- the intermediate result may be the input files that are generated for a simulation of an electronic circuit, and the key may be an MD5 or SHA cryptographic hash formed by processing all of the input files for the simulation.
- a single key of fixed size is generated regardless of the number, kinds, or sizes of objects constituting the intermediate result.
- the single key may be saved as a key in a file cache to identify the passing software test.
- the particular file or files in the file cache associated with this key are unimportant.
- one empty or nonempty file may be associated with the key.
- an empty or nonempty collection of empty or nonempty files may be associated with the key.
- the unifying idea is that the key derived from a software test's intermediate result becomes a key stored in the file cache. It will be appreciated that such a file cache may contain keys and files other than ones derived from testing results.
- a key may be generated from the intermediate result and this key may be compared with the keys saved in the file cache to identify whether the software test has already generated a passing final result.
- the generation of a final result for the software test may be bypassed because the software test has previously generated a successful final result.
- the generation of the intermediate result for a software test may be relatively fast as compared to the generation of the final result. Thus, the bypassing of generating the final result for previously passing software tests may significantly improve the efficiency of the verification of the software system.
- the key may be a 128-bit value for a key generated from the MD5 cryptographic hash function. In certain other embodiments, the key may be a 160-bit key from a cryptographic hash function in the Secure Hash Algorithm (SHA) family.
- SHA Secure Hash Algorithm
- a fixed small amount of storage may be needed to identify a software test having already generated a successful final result and this storage may be independent of the sizes, kinds, and numbers of objects constituting the intermediate and final results.
- the intermediate result is not saved in the file cache. This is particularly advantageous when the numbers and/or sizes of the objects constituting the intermediate result are large.
- the intermediate result may be stored in the file cache as a file or set of files.
- the process for verifying a software system may also automatically save the key in the file cache for software tests having successful final results, and also automatically check the key and bypass of the generation of a final result for a key that is already stored in the file cache.
- a file cache may be readily shared among multiple users, such as personnel having respective computers connected to a shared network (e.g., a corporate intranet or the Internet), the saving of a key in the file cache and the checking for the presence of a key in the file cache may be automatically shared between users.
- a shared network e.g., a corporate intranet or the Internet
- Such sharing may be expedited because a file cache may be included as a component of the software environment in which the personnel operate. As a consequence, one user's test may be abbreviated because the same test was previously completed successfully by another user.
- Certain software environments include a file cache for purposes that may be unrelated to software testing.
- a software application which is generally available to the employees in an organization, may contain a general-purpose file cache and this file cache may be available for use by other software applications.
- the general-purpose file cache may be used to abbreviate software tests despite the fact that the file cache was not intended for such usage.
- the file cache may be simultaneously used for unrelated activities.
- FIG. 1 is a flow diagram of a process for software testing in accordance with various embodiments of the invention.
- the software testing For each test in a suite of software tests, the software testing generates an intermediate result, generates a key from the intermediate result, and either generates a final result or bypasses generating a final result depending on the presence or absence of the key in a file cache.
- an intermediate result which may include multiple files or software objects, is generated for a software test.
- a test case generator automatically generates the intermediate result from parameters indicating the extent of the test suite.
- a test case generator may sometimes generate the same intermediate result multiple times.
- a single key is generated from the intermediate result.
- the presence or absence of the key in a file cache is determined. If the key is stored in the file cache, process 100 proceeds from decision 108 to step 110 , and otherwise process 100 proceeds from decision 108 to step 112 .
- the final result is generated from the intermediate result for the software test.
- the success or failure of the final result is checked. In certain embodiments, completion of the generation of the final result indicates success and the lack of completed generation of the final result indicates failure. In certain other embodiments, success or failure of the software test is a component included in the final result.
- process 100 proceeds to step 116 , and otherwise process 100 completes for the software test.
- the key generated at step 104 is stored in the file cache for the successful software test, and process 100 completes.
- step 110 the generation of the final result is bypassed and process 100 completes for a software test that previously completed successfully and consequently stores the key (from step 104 ) in the file cache at step 116 .
- the software test may have previously completed successfully because a test case generator generated the same test twice. While the possibility exists that there is a collision of differing intermediate results for two software tests that generate the same key, usage of a high-quality key generator, such as the MD5 or SHA cryptographic hash, makes such a collision almost impossible in actual practice.
- the bypassing of the generation of the final result for a previously successful software test may improve the efficiency of the software testing.
- FIG. 2 is a block diagram of a system for execution of software testing in accordance with various embodiments of the invention.
- a generator 202 generates an intermediate result 204 for a software test.
- a hash generator 206 generates a key 208 from the intermediate result 204 .
- An executor 210 checks whether the key 208 is stored in a file cache 212 . If the key 208 is not stored in the file cache 212 , the executor 210 generates a final result 214 from the intermediate result 204 , and otherwise the executor 210 bypasses the generation of the final result 214 .
- the executor 210 when the generation of the final result 214 is completed, stores the key 208 in the file cache 212 . In certain other embodiments, when the generation of the final result 214 is completed and the final result 214 indicates success of the software test, the executor 210 stores the key 208 in the file cache 212 .
- the key 208 may be stored in the file cache 212 by storing an empty file 216 in the file cache 212 under the key 208 .
- the key 208 may be stored in the file cache 212 by storing a non-empty file 216 in the file cache 212 under the key 208 .
- This non-empty file 216 may contain identifying information including one or more of a name of a user directing the software test (including generating the final result 214 ), a name of a computer performing the software test, a time and date for the software test, and a version number of at least one software module of the executor 210 that executes the software test.
- the file 216 may have a name that includes the key 208 .
- Checking for the presence of a key 208 in the file cache 212 may include comparing the key 208 with the name of a file 216 in the file cache 212 .
- Checking for the presence of a key 208 in the file cache 212 may also include attempting to retrieve a file 216 in the file cache 212 having a name corresponding to the key 208 .
- FIG. 3 is a flow diagram of a process 300 for simulating an electronic circuit in accordance with various embodiments of the invention.
- the simulation of an electronic circuit may be a software test that generates an input test vector for the electronic circuit and/or generates a specification in a hardware description language (HDL) for the electronic circuit, and simulates an HDL specification using an input test vector.
- HDL hardware description language
- an intermediate result may be generated including test vectors and/or a specification for an electronic circuit.
- the test vectors may control the stimulus for the inputs of an electronic circuit during simulation.
- a specification of an electronic circuit may also be generated.
- the software testing using simulation of a programmable logic device (PLD) may generate various electronic circuits to test the PLD that implements these electronic circuits.
- Input test vector may also be generated for each electronic circuit or fixed test vectors may instead be used for these electronic circuits.
- input test vector may be generated for each software test and the specification of the electronic circuit may be a fixed specification. It will be appreciated that the intermediate result generated at step 302 may include other input files for the simulation of the electronic circuit.
- a cryptographic hash such as the MD5 or SHA cryptographic hash, of the intermediate result, including the test vectors and/or the specification of the electronic circuit, is generated to produce a key.
- the input files of the intermediate result may be supplied to the cryptographic hash to produce a 128-bit (or larger) key that, in practice, uniquely identifies the intermediate result.
- the input files of the intermediate result may include any file or software object of the software system being used for simulation.
- the software system may include a script that controls the process of executing the simulation of the electronic circuit, and such a script may included in the intermediate result.
- a file cache is queried to determine whether the file cache contains the key of interest.
- the file cache responds to the key query by indicating whether or not the key is present.
- the file cache is queried by attempting to retrieve a file or files that are stored under the key. If any file or files have previously been stored in the file cache under the key, the file cache returns the file or files; otherwise, the file cache indicates the retrieval attempt is unsuccessful.
- Decision 308 checks the results from the query or retrieval attempt at step 306 . For a successful retrieval of one or more files, process 300 may proceed to step 310 , and for an unsuccessful retrieval attempt, process 300 proceeds to step 312 .
- a netlist is synthesized from the HDL specification of the electronic circuit.
- the netlist may be mapped to the programmable logic and routing resources of the PLD at step 314 .
- the netlist of the electronic circuit is simulated in a circuit simulator using the stimulus from the test vectors.
- the test vectors may include assertions of the expected behavior of the electronic circuit, such as assertions of the expected values observed at the outputs of the electronic circuit.
- the violation of any assertion may indicate the simulation completes with an error indicating an unsuccessful simulation. Completion of the simulation without violating an assertion and without triggering any other error detection mechanism included in the software system may indicate successful completion of the simulation.
- Step 318 checks the completion status for the simulation. For a successful completion, process 300 proceeds to step 320 ; otherwise, process 300 proceeds to step 310 .
- the key generated at step 304 is stored in the file cache by storing in the file cache a file having a name including the key.
- the simulation system may be modified to fix a defect in the simulation system used to simulate the electronic circuit.
- the defect may be in an HDL specification of the electronic circuit; however, the defect may also be in software modules of the simulation system that are used to generate simulation input.
- the generator of the test vectors may have a defect that affects test of a particular feature of the electronic circuit. After the defect is fixed in this generator of the test vectors, all the previously executed software tests may be re-executed, but only the software tests that exercise this feature of the electronic circuit may have modified test vectors. Thus, while the intermediate results for all software tests may be generated again, the only software tests that are simulated again are the software tests that have an intermediate result including modified test vectors. It will be appreciated that process 300 may complete when the simulation software system no longer needs modification at step 310 .
- a particular simulation software system may include unpredictable variations. For example, a generator of a specification of an electronic circuit may generate an input port list for a sub-circuit that may non-deterministically vary the order of the input ports in this input port list. A distinct key may be generated from the intermediate results that include each variation of the specification of the electronic circuit, and a separate simulation may be executed to generate the final results for each variation. Thus, another key for another intermediate result for a particular software test may already be stored in the file cache. However, barring other changes in the intermediate results, each variation will only be simulated once even though certain of the variations are generated multiple times.
- FIG. 4 is a block diagram of a system 400 for software testing in accordance with various embodiments of the invention.
- the processor-readable device 402 may be a program storage medium, including various modules 404 through 414 for software testing.
- the modules 404 through 414 may be read from processor-readable device 402 by processor 416 and instructions in each module may be executed by processor 416 .
- Execution of the instructions of module 404 through 414 by processor 416 causes the processor 416 to perform the designated operations.
- Execution of the instructions of module 404 causes processor 416 to generate an intermediate result for a software test and execution of the instructions of module 406 causes processor 416 to generate a key from the intermediate result.
- the processor 416 determines whether the key is stored in a file cache.
- the instructions of module 410 cause processor 416 to generate a final result for the software test in response to the key not being stored in the file cache, and execution of the instructions of module 414 causes processor 416 to bypass generation of the final result in response to the key being stored in the file cache.
- the processor 416 in executing the instructions of module 410 , stores the key in the file cache in response to successful completion of the generation of the final result.
- FIG. 5 is a block diagram of a programmable logic device (PLD) that may implement an electronic circuit that is simulated in accordance with various embodiments of the invention.
- PLDs programmable logic device
- FIG. 5 illustrates an FPGA architecture 500 that includes a large number of different programmable tiles of programmable logic and routing resources including multi-gigabit transceivers (MGTs 501 ), configurable logic blocks (CLBs 502 ), random access memory blocks (BRAMs 503 ), input/output blocks (IOBs 504 ), configuration and clocking logic (CONFIG/CLOCKS 505 ), digital signal processing blocks (DSPs 506 ), specialized input/output blocks (I/O 507 ) (e.g., configuration ports and clock ports), and other programmable logic 508 such as digital clock managers, analog-to-digital converters, system monitoring logic, and so forth.
- Some FPGAs also include dedicated processor blocks (PROC 510 ).
- each programmable tile includes a programmable interconnect element (INT 511 ) having standardized connections to and from a corresponding interconnect element in each adjacent tile. Therefore, the programmable interconnect elements taken together implement the programmable interconnect structure for the illustrated FPGA.
- the programmable interconnect element (INT 511 ) also includes the connections to and from the programmable logic element within the same tile, as shown by the examples included at the top of FIG. 5 .
- a CLB 502 can include a configurable logic element (CLE 512 ) that can be programmed to implement user logic plus a single programmable interconnect element (INT 511 ).
- a BRAM 503 can include a BRAM logic element (BRL 513 ) in addition to one or more programmable interconnect elements.
- BRAM logic element BRAM logic element
- the number of interconnect elements included in a tile depends on the height of the tile. In the pictured embodiment, a BRAM tile has the same height as four CLBs, but other numbers (e.g., five) can also be used.
- a DSP tile 506 can include a DSP logic element (DSPL 514 ) in addition to an appropriate number of programmable interconnect elements.
- An IOB 504 can include, for example, two instances of an input/output logic element (IOL 515 ) in addition to one instance of the programmable interconnect element (INT 511 ).
- IOL 515 input/output logic element
- INT 511 programmable interconnect element
- a columnar area near the center of the die (shown shaded in FIG. 5 ) is used for configuration, clock, and other control logic. Horizontal areas 509 extending from this column are used to distribute the clocks and configuration signals across the breadth of the FPGA.
- Some FPGAs utilizing the architecture illustrated in FIG. 5 include additional logic blocks that disrupt the regular columnar structure making up a large part of the FPGA.
- the additional logic blocks can be programmable blocks and/or dedicated logic.
- the processor block PROC 510 shown in FIG. 5 spans several columns of CLBs and BRAMs.
- FIG. 5 is intended to illustrate only an exemplary FPGA architecture.
- the numbers of logic blocks in a column, the relative widths of the columns, the number and order of columns, the types of logic blocks included in the columns, the relative sizes of the logic blocks, and the interconnect/logic implementations included at the top of FIG. 5 are purely exemplary.
- more than one adjacent column of CLBs is typically included wherever the CLBs appear, to facilitate the efficient implementation of user logic.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
Description
Claims (14)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/264,732 US7673288B1 (en) | 2005-11-01 | 2005-11-01 | Bypassing execution of a software test using a file cache |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/264,732 US7673288B1 (en) | 2005-11-01 | 2005-11-01 | Bypassing execution of a software test using a file cache |
Publications (1)
Publication Number | Publication Date |
---|---|
US7673288B1 true US7673288B1 (en) | 2010-03-02 |
Family
ID=41717762
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/264,732 Expired - Fee Related US7673288B1 (en) | 2005-11-01 | 2005-11-01 | Bypassing execution of a software test using a file cache |
Country Status (1)
Country | Link |
---|---|
US (1) | US7673288B1 (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110054643A1 (en) * | 2009-08-26 | 2011-03-03 | Gary Keith Law | Methods and apparatus to manage testing of a process control system |
US20130239091A1 (en) * | 2010-03-19 | 2013-09-12 | Ebay Inc. | Orthogonal experimentation in a computing environment |
US20150234733A1 (en) * | 2014-02-18 | 2015-08-20 | International Business Machines Corporation | Software testing |
US10210064B2 (en) * | 2015-08-27 | 2019-02-19 | Google Llc | Systems and methods for device compatibility testing and reporting |
US11048618B2 (en) * | 2019-03-11 | 2021-06-29 | International Business Machines Corporation | Environment modification for software application testing |
US11086963B2 (en) | 2018-12-05 | 2021-08-10 | Ebay Inc. | Adaptive data platforms |
US11200154B2 (en) | 2019-03-11 | 2021-12-14 | International Business Machines Corporation | Function modification for software application testing |
Citations (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5633813A (en) * | 1994-05-04 | 1997-05-27 | Srinivasan; Seshan R. | Apparatus and method for automatic test generation and fault simulation of electronic circuits, based on programmable logic circuits |
US5673387A (en) * | 1994-05-16 | 1997-09-30 | Lucent Technologies Inc. | System and method for selecting test units to be re-run in software regression testing |
US6167545A (en) * | 1998-03-19 | 2000-12-26 | Xilinx, Inc. | Self-adaptive test program |
US6223272B1 (en) * | 1998-07-15 | 2001-04-24 | Siemens Aktiengesellschaft | Test vector verification system |
US6292569B1 (en) * | 1996-08-12 | 2001-09-18 | Intertrust Technologies Corp. | Systems and methods using cryptography to protect secure computing environments |
US6367013B1 (en) * | 1995-01-17 | 2002-04-02 | Eoriginal Inc. | System and method for electronic transmission, storage, and retrieval of authenticated electronic original documents |
US6490696B1 (en) * | 1999-12-15 | 2002-12-03 | Electronics For Imaging, Inc. | System and method for printer output regression testing using display lists |
US6594820B1 (en) * | 1999-09-28 | 2003-07-15 | Sun Microsystems, Inc. | Method and apparatus for testing a process in a computer system |
US20030204784A1 (en) * | 2002-04-29 | 2003-10-30 | Jorapur Gopal P. | System and method for automatic test case generation |
US20040177332A1 (en) * | 2003-03-07 | 2004-09-09 | Manish Pandey | Method and system for logic equivalence checking |
US6816952B1 (en) * | 2002-05-31 | 2004-11-09 | Unisys Corporation | Lock management system and method for use in a data processing system |
US20050172267A1 (en) * | 2004-01-30 | 2005-08-04 | Derek Bergin | Method and system for testing software |
US20050289355A1 (en) * | 2004-06-29 | 2005-12-29 | Dimitri Kitariev | Lockstep mechanism to ensure security in hardware at power-up |
US20060047861A1 (en) * | 2004-08-27 | 2006-03-02 | Incomm Technologies Co., Ltd. | Method for soft configuring communication protocols |
US7225416B1 (en) * | 2004-06-15 | 2007-05-29 | Altera Corporation | Methods and apparatus for automatic test component generation and inclusion into simulation testbench |
US7363618B2 (en) * | 2001-02-14 | 2008-04-22 | International Business Machines Corporation | Software testing |
-
2005
- 2005-11-01 US US11/264,732 patent/US7673288B1/en not_active Expired - Fee Related
Patent Citations (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5633813A (en) * | 1994-05-04 | 1997-05-27 | Srinivasan; Seshan R. | Apparatus and method for automatic test generation and fault simulation of electronic circuits, based on programmable logic circuits |
US5673387A (en) * | 1994-05-16 | 1997-09-30 | Lucent Technologies Inc. | System and method for selecting test units to be re-run in software regression testing |
US6367013B1 (en) * | 1995-01-17 | 2002-04-02 | Eoriginal Inc. | System and method for electronic transmission, storage, and retrieval of authenticated electronic original documents |
US6292569B1 (en) * | 1996-08-12 | 2001-09-18 | Intertrust Technologies Corp. | Systems and methods using cryptography to protect secure computing environments |
US6167545A (en) * | 1998-03-19 | 2000-12-26 | Xilinx, Inc. | Self-adaptive test program |
US6223272B1 (en) * | 1998-07-15 | 2001-04-24 | Siemens Aktiengesellschaft | Test vector verification system |
US6594820B1 (en) * | 1999-09-28 | 2003-07-15 | Sun Microsystems, Inc. | Method and apparatus for testing a process in a computer system |
US6490696B1 (en) * | 1999-12-15 | 2002-12-03 | Electronics For Imaging, Inc. | System and method for printer output regression testing using display lists |
US7363618B2 (en) * | 2001-02-14 | 2008-04-22 | International Business Machines Corporation | Software testing |
US20030204784A1 (en) * | 2002-04-29 | 2003-10-30 | Jorapur Gopal P. | System and method for automatic test case generation |
US6816952B1 (en) * | 2002-05-31 | 2004-11-09 | Unisys Corporation | Lock management system and method for use in a data processing system |
US20040177332A1 (en) * | 2003-03-07 | 2004-09-09 | Manish Pandey | Method and system for logic equivalence checking |
US20050172267A1 (en) * | 2004-01-30 | 2005-08-04 | Derek Bergin | Method and system for testing software |
US7225416B1 (en) * | 2004-06-15 | 2007-05-29 | Altera Corporation | Methods and apparatus for automatic test component generation and inclusion into simulation testbench |
US20050289355A1 (en) * | 2004-06-29 | 2005-12-29 | Dimitri Kitariev | Lockstep mechanism to ensure security in hardware at power-up |
US20060047861A1 (en) * | 2004-08-27 | 2006-03-02 | Incomm Technologies Co., Ltd. | Method for soft configuring communication protocols |
Cited By (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9874870B2 (en) * | 2009-08-26 | 2018-01-23 | Fisher-Rosemount Systems, Inc. | Methods and apparatus to manage testing of a process control system |
US20110054643A1 (en) * | 2009-08-26 | 2011-03-03 | Gary Keith Law | Methods and apparatus to manage testing of a process control system |
US20190196941A1 (en) * | 2010-03-19 | 2019-06-27 | Ebay Inc. | Orthogonal experimentation in a computing environment |
US20130239091A1 (en) * | 2010-03-19 | 2013-09-12 | Ebay Inc. | Orthogonal experimentation in a computing environment |
US10983900B2 (en) * | 2010-03-19 | 2021-04-20 | Ebay Inc. | Orthogonal experimentation in a computing environment |
US9262543B2 (en) * | 2010-03-19 | 2016-02-16 | Ebay Inc. | Orthogonal experimentation in a computing environment |
US20160162390A1 (en) * | 2010-03-19 | 2016-06-09 | Ebay Inc. | Orthogonal experimentation in a computing environment |
US20140297795A1 (en) * | 2010-03-19 | 2014-10-02 | Ebay Inc. | Orthogonal experimentation in a computing environment |
US9703685B2 (en) * | 2010-03-19 | 2017-07-11 | Ebay Inc. | Orthogonal experimentation in a computing environment |
US8789019B2 (en) * | 2010-03-19 | 2014-07-22 | Ebay Inc. | Orthogonal experimentation in a computing environment |
US10599557B2 (en) * | 2010-03-19 | 2020-03-24 | Ebay Inc. | Orthogonal experimentation in a computing environment |
US10268569B2 (en) * | 2010-03-19 | 2019-04-23 | Ebay Inc. | Orthogonal experimentation in a computing environment |
US9632917B2 (en) * | 2014-02-18 | 2017-04-25 | International Business Machines Corporation | Software testing |
US20150234733A1 (en) * | 2014-02-18 | 2015-08-20 | International Business Machines Corporation | Software testing |
US10210064B2 (en) * | 2015-08-27 | 2019-02-19 | Google Llc | Systems and methods for device compatibility testing and reporting |
US11086963B2 (en) | 2018-12-05 | 2021-08-10 | Ebay Inc. | Adaptive data platforms |
US11921811B2 (en) | 2018-12-05 | 2024-03-05 | Ebay Inc. | Adaptive data platforms |
US11048618B2 (en) * | 2019-03-11 | 2021-06-29 | International Business Machines Corporation | Environment modification for software application testing |
US11200154B2 (en) | 2019-03-11 | 2021-12-14 | International Business Machines Corporation | Function modification for software application testing |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6931611B2 (en) | Design verification system for avoiding false failures and method therefor | |
US7315973B1 (en) | Method and apparatus for choosing tests for simulation and associated algorithms and hierarchical bipartite graph data structure | |
US7673288B1 (en) | Bypassing execution of a software test using a file cache | |
US6993470B2 (en) | Method of evaluating test cases in a simulation environment by harvesting | |
US8892386B2 (en) | Method and apparatus for post-silicon testing | |
TWI468936B (en) | A system and method for the generation of verification programs | |
US8990622B2 (en) | Post-silicon validation using a partial reference model | |
CN112417798B (en) | Time sequence testing method and device, electronic equipment and storage medium | |
US20080092004A1 (en) | Method and system for automated path delay test vector generation from functional tests | |
US8762907B2 (en) | Hierarchical equivalence checking and efficient handling of equivalence checks when engineering change orders are in an unsharable register transfer level | |
US8060847B2 (en) | Clock model for formal verification of a digital circuit description | |
JP2009230451A (en) | Equivalence verifying method, equivalence verification program and method for generating equivalence verification program | |
JPH05505271A (en) | How to test and debug computer programs | |
US7502966B2 (en) | Testcase generation via a pool of parameter files | |
US9721058B2 (en) | System and method for reactive initialization based formal verification of electronic logic design | |
US9058452B1 (en) | Systems and methods for tracing and fixing unknowns in gate-level simulation | |
EP1327890A2 (en) | Integrated circuit defect detection system | |
Maniatakos et al. | AVF analysis acceleration via hierarchical fault pruning | |
US20040006751A1 (en) | System verifying apparatus and method | |
US8056037B2 (en) | Method for validating logical function and timing behavior of a digital circuit decision | |
US7051301B2 (en) | System and method for building a test case including a summary of instructions | |
US7133818B2 (en) | Method and apparatus for accelerated post-silicon testing and random number generation | |
US7650579B2 (en) | Model correspondence method and device | |
Suryasarman et al. | Rsbst: an accelerated automated software-based self-test synthesis for processor testing | |
Refan et al. | Bridging presilicon and postsilicon debugging by instruction-based trace signal selection in modern processors |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: XILINX, INC.,CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:STROOMER, JEFFREY D.;REEL/FRAME:017185/0844 Effective date: 20051027 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552) Year of fee payment: 8 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20220302 |