Package csb :: Package test
[frames] | no frames]

Source Code for Package csb.test

  1  """ 
  2  This is a top level package, hosting the entire CSB test framework. It is divided 
  3  into several major parts: 
  4   
  5      - test cases, located under csb.test.cases 
  6      - test data, in C{/csb/test/data} (not a package) 
  7      - test console, in C{/csb/test/app.py} 
  8   
  9  This module, csb.test, contains all the glue-code functions, classes and  
 10  decorators you would need in order to write tests for CSB.     
 11   
 12      1. Configuration and Tree 
 13       
 14         L{Config<csb.test.Config>} is a common config object shared between CSB 
 15         tests. Each config instance contains properties like: 
 16               
 17              - data: the data folder, automatically discovered and loaded in 
 18                csb.test.Config.DATA at module import time 
 19              - temp: a default temp folder, which test cases can use 
 20           
 21         Each L{Config<csb.test.Config>} provides a convenient way to retrieve 
 22         files from C{/csb/test/data}. Be sure to check out L{Config.getTestFile} 
 23         and L{Config.getPickle}. In case you need a temp file, use 
 24         L{Config.getTempStream} or have a look at L{csb.io.TempFile} and 
 25         L{csb.io.TempFolder}.  
 26           
 27         All test data files should be placed in the C{data} folder. All test 
 28         modules must be placed in the root package: csb.test.cases. There is 
 29         a strict naming convention for test modules: the name of a test module 
 30         should be the same as the name of the CSB API package it tests. For  
 31         example, if you are writing tests for C{csb/bio/io/__init__.py}, the 
 32         test module must be C{csb/test/cases/bio/io/__init__.py}. C{csb.test.cases} 
 33         is the root package of all test modules in CSB. 
 34       
 35      2. Writing Tests 
 36       
 37         Writing a test is easy. All you need is to import csb.test and then 
 38         create your own test cases, derived from L{csb.test.Case}: 
 39          
 40             >>> import csb.test 
 41             >>> @csb.test.unit 
 42                 class TestSomeClass(csb.test.Case): 
 43                     def setUp(self): 
 44                         super(TestSomeClass, self).setUp() 
 45                         # do something with self.config here... 
 46          
 47         In this way your test case instance is automatically equipped with a  
 48         reference to the test config, so your test method can be: 
 49   
 50             >>> @csb.test.unit 
 51                 class TestSomeClass(csb.test.Case): 
 52                     def testSomeMethod(self): 
 53                         myDataFile = self.config.getTestFile('some.file') 
 54                         self.assert... 
 55           
 56         The "unit" decorator marks a test case as a collection of unit tests. 
 57         All possibilities are: L{csb.test.unit}, L{csb.test.functional}, L{csb.test.custom}, 
 58         and L{csb.test.regression}. 
 59                      
 60         Writing custom (a.k.a. "data", "slow", "dynamic") tests is a little bit 
 61         more work. Custom tests must be functions, not classes. Basically a 
 62         custom test is a function, which builds a unittest.TestSuite instance  
 63         and then returns it when called without arguments. 
 64          
 65         Regression tests are usually created in response to reported bugs. Therefore,  
 66         the best practice is to mark each test method with its relevant bug ID: 
 67          
 68             >>> @csb.test.regression 
 69                 class SomeClassRegressions(csb.test.Case) 
 70                     def testSomeFeature(self) 
 71                     \""" 
 72                     @see: [CSB 000XXXX]  
 73                     \""" 
 74                     # regression test body... 
 75              
 76      3. Style Guide: 
 77       
 78         - name test case packages as already described 
 79         - group tests in csb.test.Case-s and name them properly 
 80         - prefix test methods with "test", like "testParser" - very important 
 81         - use camelCase for methods and variables. This applies to all the 
 82           code under csb.test (including test) and does not apply to the rest 
 83           of the library! 
 84         - for functional tests it's okay to define just one test method: runTest 
 85         - for unit tests you should create more specific test names, for example:  
 86           "testParseFile" - a unit test for some method called "parse_file" 
 87         - use csb.test decorators to mark tests as unit, functional, regression, etc. 
 88         - make every test module executable:: 
 89          
 90             if __name__ == '__main__': 
 91                 csb.test.Console()   # Discovers and runs all test cases in the module 
 92       
 93      4. Test Execution 
 94       
 95         Test discovery is handled by C{test builders} and a test runner 
 96         C{app}. Test builders are subclasses of L{AbstractTestBuilder}.   
 97         For every test type (unit, functional, regression, custom) there is a 
 98         corresponding test builder. L{AnyTestBuilder} is a special builder which 
 99         scans for unit, regression and functional tests at the same time. 
100   
101         Test builder classes inherit the following test discovery methods: 
102       
103             - C{loadTests} - load tests from a test namespace. Wildcard 
104               namespaces are handled by C{loadAllTests} 
105             - C{loadAllTests} - load tests from the given namespace, and 
106               from all sub-packages (recursive) 
107             - C{loadFromFile} - load tests from an absolute file name 
108             - C{loadMultipleTests} - calls C{loadTests} for a list of  
109               namespaces and combines all loaded tests in a single suite 
110                
111         Each of those return test suite objects, which can be directly executed 
112         with python's unittest runner. 
113          
114         Much simpler way to execute a test suite is to use our test app  
115         (C{csb/test/app.py}), which is simply an instance of L{csb.test.Console}:: 
116          
117             $ python csb/test/app.py --help 
118          
119         The app has two main arguments:  
120       
121             - test type - tells the app which TestBuilder to use for test dicsovery 
122               ("any" triggers L{AnyTestBuilder}, "unit" - L{UnitTestBuilder}, etc.)  
123             - test namespaces - a list of "dotted" test modules, for example:: 
124       
125                  csb.test.cases.bio.io.*   # io and sub-packages 
126                  csb.test.cases.bio.utils  # only utils 
127                  .                         # current module 
128       
129         In addition to running the app from the command line, you can run it 
130         also programmatically by instantiating L{csb.test.Console}. You can 
131         construct a test console object by passing a list of test namespace(s) 
132         and a test builder class to the Console's constructor. 
133   
134       
135      5. Commit Policies 
136       
137         Follow these guidelines when making changes to the repository: 
138       
139             - B{no bugs in "trunk"}: after fixing a bug or implementing a new 
140               feature, make sure at least the default test set passes by running 
141               the test console without any arguments. This is equivalent to: 
142               app.py -t any "csb.test.cases.*". (If no test case from this set covers 
143               the affected code, create a test case first, as described in the other 
144               policies) 
145       
146             - B{no recurrent issues}: when a bug is found, first write a regression 
147               test with a proper "@see: BugID" tag in the docstring. Run the test 
148               to make sure it fails. After fixing the bug, run the test again before 
149               you commit, as required by the previous policy 
150                
151             - B{test all new features}: there should be a test case for every new feature 
152               we implement. One possible approach is to write a test case first and 
153               make sure it fails; when the new feature is ready, run the test again 
154               to make sure it passes 
155   
156  @warning: for compatibility reasons do NOT import and use the unittest module 
157            directly. Always import unittest from csb.test, which is guaranteed 
158            to be python 2.7+ compatible. The standard unittest under python 2.6 
159            is missing some features, that's why csb.test will take care of 
160            replacing it with unittest2 instead.  
161  """ 
162  import os 
163  import sys 
164  import imp 
165  import types 
166  import time 
167  import tempfile 
168  import traceback 
169  import argparse 
170   
171  import csb.io 
172  import csb.core 
173   
174  try: 
175      from unittest import skip, skipIf 
176      import unittest 
177  except ImportError: 
178      import unittest2 as unittest 
179   
180  from abc import ABCMeta, abstractproperty 
181 182 183 -class Attributes(object):
184 185 UNIT = '__CSBUnitTest__' 186 CUSTOM = '__CSBCustomTest__' 187 FUNCTIONAL = '__CSBFunctionalTest__' 188 REGRESSION = '__CSBRegressionTest__'
189
190 -class Config(object):
191 """ 192 General CSB Test Config. Config instances contain the following properties: 193 194 - data - path to the CSB Test Data directory. Default is L{Config.DATA} 195 - temp - path to the system's temp directory. Default is L{Config.TEMP} 196 - config - the L{Config} class 197 """ 198 199 DATA = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'data') 200 """ 201 @cvar: path to the default test data directory: <install dir>/csb/test/data 202 """ 203 GENERATED_DATA = DATA 204 """ 205 @cvar: path to the default data directory for generated test files 206 """ 207 TEMP = os.path.abspath(tempfile.gettempdir()) 208 """ 209 @cvar: path to the default system's temp directory 210 """ 211 212 @staticmethod
213 - def setDefaultDataRoot(path):
214 """ 215 Override the default L{Config.DATA} with a new data root directory. 216 217 @param path: full directory path 218 @type path: str 219 """ 220 if not os.path.isdir(path): 221 raise IOError('Path not found: {0}'.format(path)) 222 223 Config.DATA = os.path.abspath(path)
224 225 @staticmethod
227 """ 228 Override the default L{Config.GENERATED_DATA} with a new data root directory. 229 230 @param path: full directory path 231 @type path: str 232 """ 233 if not os.path.isdir(path): 234 raise IOError('Path not found: {0}'.format(path)) 235 236 Config.GENERATED_DATA = os.path.abspath(path)
237 238 @property
239 - def data(self):
240 """ 241 Test data directory 242 @rtype: str 243 """ 244 return Config.DATA
245 246 @property
247 - def generated_data(self):
248 """ 249 Test data directory for generated files 250 @rtype: str 251 """ 252 return Config.GENERATED_DATA
253 254 @property
255 - def temp(self):
256 """ 257 Test temp directory 258 @rtype: str 259 """ 260 return Config.TEMP
261
262 - def getTestFile(self, fileName, subDir=''):
263 """ 264 Search for C{fileName} in the L{Config.DATA} directory. If not found, 265 try also L{Config.GENERATED_DATA} (if different). 266 267 @param fileName: the name of a test file to retrieve 268 @type fileName: str 269 @param subDir: scan a sub-directory of L{Config.DATA} 270 @type subDir: str 271 272 @return: full path to C{fileName} 273 @rtype: str 274 275 @raise IOError: if no such file is found 276 """ 277 for data in [self.data, self.generated_data]: 278 file = os.path.join(data, subDir, fileName) 279 280 if os.path.isfile(file): 281 return file 282 283 raise IOError('Test file not found: {0}'.format(fileName))
284
285 - def getPickle(self, fileName, subDir=''):
286 """ 287 Same as C{self.getTestFile}, but try to unpickle the the file 288 and return the unpickled object. Pickles are usually stored in 289 L{Config.GENERATED_DATA}. 290 291 @param fileName: the name of a test file to retrieve 292 @type fileName: str 293 @param subDir: scan a sub-directory of L{Config.DATA} 294 @type subDir: str 295 296 @rtype: object 297 """ 298 file = self.getTestFile(fileName, subDir) 299 return csb.io.Pickle.load(open(file, 'rb'))
300
301 - def getContent(self, fileName, subDir=''):
302 """ 303 Same as C{self.getTestFile}, but also read and return the contents of 304 the file. 305 306 @param fileName: the name of a test file to retrieve 307 @type fileName: str 308 @param subDir: scan a sub-directory of L{Config.DATA} 309 @type subDir: str 310 311 @rtype: str 312 """ 313 with open(self.getTestFile(fileName, subDir)) as f: 314 return f.read()
315
316 - def getTempStream(self, mode='t'):
317 """ 318 Return a temporary file stream:: 319 320 with self.getTempStream() as tmp: 321 tmp.write(something) 322 tmp.flush() 323 file_name = tmp.name 324 325 @param mode: file open mode (text, binary), default=t 326 @type mode: str 327 @rtype: file stream 328 """ 329 return csb.io.TempFile(mode=mode)
330
331 - def ensureDataConsistency(self):
332 """ 333 Try to deserialize some pickled data files. Call L{Config.updateDataFiles} 334 if the pickles appeared incompatible with the current interpreter. 335 """ 336 try: 337 self.getPickle('1nz9.model1.pickle') 338 except: 339 self.updateDataFiles()
340
341 - def updateDataFiles(self):
342 """ 343 Refresh the pickled structures in csb/test/data. This might be needed when 344 the internal representation of some classes has changed. 345 """ 346 from csb.io import Pickle 347 from csb.bio.io.wwpdb import RegularStructureParser 348 from csb.bio.structure import Ensemble, ChemElements 349 350 parser = RegularStructureParser(self.getTestFile('1nz9.pdb')) 351 model1 = parser.parse_structure(model=1) 352 model2 = parser.parse_structure(model=2) 353 354 ensemble = Ensemble() 355 ensemble.models.append(model1) 356 ensemble.models.append(model2) 357 Pickle.dump(ensemble, open(os.path.join(self.generated_data, '1nz9.full.pickle'), 'wb')) 358 359 mse = model1.chains['A'].find(164) 360 mse.label = 'MSE' 361 mse.atoms['SD']._element = ChemElements.Se 362 mse.atoms['SD']._full_name = 'SE ' 363 Pickle.dump(model1, open(os.path.join(self.generated_data, '1nz9.model1.pickle'), 'wb'))
364
365 -class Case(unittest.TestCase):
366 """ 367 Base class, defining a CSB Test Case. Provides a default implementation 368 of C{unittest.TestCase.setUp} which grabs a reference to a L{Config}. 369 """ 370 371 @property
372 - def config(self):
373 """ 374 Test config instance 375 @rtype: L{Config} 376 """ 377 return self.__config
378
379 - def setUp(self):
380 """ 381 Provide a reference to the CSB Test Config in the C{self.config} property. 382 """ 383 self.__config = Config() 384 assert hasattr(self.config, 'data'), 'The CSB Test Config must contain the data directory' 385 assert self.config.data, 'The CSB Test Config must contain the data directory'
386
387 - def reRaise(self, addArgs=()):
388 """ 389 Re-raise the last exception with its full traceback, but modify the 390 argument list with C{addArgs} and the original stack trace. 391 392 @param addArgs: additional arguments to append to the exception 393 @type addArgs: tuple 394 """ 395 klass, ex, _tb = sys.exc_info() 396 ex.args = list(ex.args) + list(addArgs) + [''.join(traceback.format_exc())] 397 398 raise klass(ex.args)
399
400 - def assertAlmostEqual(self, first, second, places=None, msg=None, delta=None):
401 402 if first == second: 403 return 404 if delta is not None and places is not None: 405 raise TypeError("specify delta or places not both") 406 407 if delta is not None: 408 409 if abs(first - second) <= delta: 410 return 411 412 m = '{0} != {1} within {2} delta'.format(first, second, delta) 413 msg = self._formatMessage(msg, m) 414 415 raise self.failureException(msg) 416 417 else: 418 if places is None: 419 places = 7 420 421 return super(Case, self).assertAlmostEqual(first, second, places=places, msg=msg)
422
423 - def assertFasterThan(self, duration, callable, *args, **kargs):
424 """ 425 Fail if it took more than C{duration} seconds to invoke C{callable}. 426 427 @param duration: maximum amount of seconds allowed 428 @type duration: float 429 """ 430 431 start = time.time() 432 callable(*args, **kargs) 433 execution = time.time() - start 434 435 if execution > duration: 436 self.fail('{0}s is slower than {1}s)'.format(execution, duration))
437 438 @classmethod
439 - def execute(cls):
440 """ 441 Run this test case. 442 """ 443 suite = unittest.TestLoader().loadTestsFromTestCase(cls) 444 runner = unittest.TextTestRunner() 445 446 return runner.run(suite)
447
448 -class InvalidNamespaceError(NameError, ImportError):
449 pass
450
451 -class AbstractTestBuilder(object):
452 """ 453 This is a base class, defining a test loader which exposes the C{loadTests} 454 method. 455 456 Subclasses must override the C{labels} abstract property, which controls 457 what kind of test cases are loaded by the test builder. 458 """ 459 460 __metaclass__ = ABCMeta 461 462 @abstractproperty
463 - def labels(self):
464 pass
465
466 - def loadFromFile(self, file):
467 """ 468 Load L{csb.test.Case}s from a module file. 469 470 @param file: test module file name 471 @type file: str 472 473 @return: a C{unittest.TestSuite} ready for the test runner 474 @rtype: C{unittest.TestSuite} 475 """ 476 mod = self._loadSource(file) 477 suite = unittest.TestLoader().loadTestsFromModule(mod) 478 return unittest.TestSuite(self._filter(suite))
479
480 - def loadTests(self, namespace):
481 """ 482 Load L{csb.test.Case}s from the given CSB C{namespace}. If the namespace 483 ends with a wildcard, tests from sub-packages will be loaded as well. 484 If the namespace is '__main__' or '.', tests are loaded from __main__. 485 486 @param namespace: test module namespace, e.g. 'csb.test.cases.bio' will 487 load tests from '/csb/test/cases/bio/__init__.py' 488 @type namespace: str 489 490 @return: a C{unittest.TestSuite} ready for the test runner 491 @rtype: C{unittest.TestSuite} 492 """ 493 if namespace.strip() == '.*': 494 namespace = '__main__.*' 495 elif namespace.strip() == '.': 496 namespace = '__main__' 497 498 if namespace.endswith('.*'): 499 return self.loadAllTests(namespace[:-2]) 500 else: 501 loader = unittest.TestLoader() 502 tests = loader.loadTestsFromName(namespace) 503 return unittest.TestSuite(self._filter(tests))
504
505 - def loadMultipleTests(self, namespaces):
506 """ 507 Load L{csb.test.Case}s from a list of given CSB C{namespaces}. 508 509 @param namespaces: a list of test module namespaces, e.g. 510 ('csb.test.cases.bio', 'csb.test.cases.bio.io') will 511 load tests from '/csb/test/cases/bio.py' and 512 '/csb/test/cases/bio/io.py' 513 @type namespaces: tuple of str 514 515 @return: a C{unittest.TestSuite} ready for the test runner 516 @rtype: C{unittest.TestSuite} 517 """ 518 if not csb.core.iterable(namespaces): 519 raise TypeError(namespaces) 520 521 return unittest.TestSuite(self.loadTests(n) for n in namespaces)
522
523 - def loadAllTests(self, namespace, extension='.py'):
524 """ 525 Load L{csb.test.Case}s recursively from the given CSB C{namespace} and 526 all of its sub-packages. Same as:: 527 528 builder.loadTests('namespace.*') 529 530 @param namespace: test module namespace, e.g. 'csb.test.cases.bio' will 531 load tests from /csb/test/cases/bio/*' 532 @type namespace: str 533 534 @return: a C{unittest.TestSuite} ready for the test runner 535 @rtype: C{unittest.TestSuite} 536 """ 537 suites = [] 538 539 try: 540 base = __import__(namespace, level=0, fromlist=['']).__file__ 541 except ImportError: 542 raise InvalidNamespaceError('Namespapce {0} is not importable'.format(namespace)) 543 544 if os.path.splitext(os.path.basename(base))[0] != '__init__': 545 suites.append(self.loadTests(namespace)) 546 547 else: 548 549 for entry in os.walk(os.path.dirname(base)): 550 551 for item in entry[2]: 552 file = os.path.join(entry[0], item) 553 if extension and item.endswith(extension): 554 suites.append(self.loadFromFile(file)) 555 556 return unittest.TestSuite(suites)
557
558 - def _loadSource(self, path):
559 """ 560 Import and return the Python module identified by C{path}. 561 562 @note: Module objects behave as singletons. If you import two different 563 modules and give them the same name in imp.load_source(mn), this 564 counts for a redefinition of the module originally named mn, which 565 is basically the same as reload(mn). Therefore, you need to ensure 566 that for every call to imp.load_source(mn, src.py) the mn parameter 567 is a string that uniquely identifies the source file src.py. 568 """ 569 name = os.path.splitext(os.path.abspath(path))[0] 570 name = name.replace('.', '-').rstrip('__init__').strip(os.path.sep) 571 572 return imp.load_source(name, path)
573
574 - def _recurse(self, obj):
575 """ 576 Extract test cases recursively from a test C{obj} container. 577 """ 578 cases = [] 579 if isinstance(obj, unittest.TestSuite) or csb.core.iterable(obj): 580 for item in obj: 581 cases.extend(self._recurse(item)) 582 else: 583 cases.append(obj) 584 return cases
585
586 - def _filter(self, tests):
587 """ 588 Filter a list of objects using C{self.labels}. 589 """ 590 filtered = [] 591 592 for test in self._recurse(tests): 593 for label in self.labels: 594 if hasattr(test, label) and getattr(test, label) is True: 595 filtered.append(test) 596 597 return filtered
598
599 -class AnyTestBuilder(AbstractTestBuilder):
600 """ 601 Build a test suite of cases, marked as either unit, functional or regression 602 tests. For detailed documentation see L{AbstractTestBuilder}. 603 """ 604 @property
605 - def labels(self):
607
608 -class UnitTestBuilder(AbstractTestBuilder):
609 """ 610 Build a test suite of cases, marked as unit tests. 611 For detailed documentation see L{AbstractTestBuilder}. 612 """ 613 @property
614 - def labels(self):
615 return [Attributes.UNIT]
616
617 -class FunctionalTestBuilder(AbstractTestBuilder):
618 """ 619 Build a test suite of cases, marked as functional tests. 620 For detailed documentation see L{AbstractTestBuilder}. 621 """ 622 @property
623 - def labels(self):
624 return [Attributes.FUNCTIONAL]
625
626 -class RegressionTestBuilder(AbstractTestBuilder):
627 """ 628 Build a test suite of cases, marked as regression tests. 629 For detailed documentation see L{AbstractTestBuilder}. 630 """ 631 @property
632 - def labels(self):
633 return [Attributes.REGRESSION]
634
635 -class CustomTestBuilder(AbstractTestBuilder):
636 """ 637 Build a test suite of cases, marked as custom tests. CustomTestBuilder will 638 search for functions, marked with the 'custom' test decorator, which return 639 a dynamically built C{unittest.TestSuite} object when called without 640 parameters. This is convenient when doing data-related tests, e.g. 641 instantiating a single type of a test case many times iteratively, for 642 each entry in a database. 643 644 For detailed documentation see L{AbstractTestBuilder}. 645 """ 646 @property
647 - def labels(self):
648 return [Attributes.CUSTOM]
649
650 - def loadFromFile(self, file):
651 652 mod = self._loadSource(file) 653 suites = self._inspect(mod) 654 655 return unittest.TestSuite(suites)
656
657 - def loadTests(self, namespace):
658 659 if namespace.strip() == '.*': 660 namespace = '__main__.*' 661 elif namespace.strip() == '.': 662 namespace = '__main__' 663 664 if namespace.endswith('.*'): 665 return self.loadAllTests(namespace[:-2]) 666 else: 667 try: 668 mod = __import__(namespace, fromlist=['']) 669 except ImportError: 670 raise InvalidNamespaceError('Namespace {0} is not importable'.format(namespace)) 671 suites = self._inspect(mod) 672 return unittest.TestSuite(suites)
673
674 - def _inspect(self, module):
675 676 objects = map(lambda n: getattr(module, n), dir(module)) 677 return self._filter(objects)
678
679 - def _filter(self, factories):
680 """ 681 Filter a list of objects using C{self.labels}. 682 """ 683 filtered = [] 684 685 for obj in factories: 686 for label in self.labels: 687 if hasattr(obj, label) and getattr(obj, label) is True: 688 suite = obj() 689 if not isinstance(suite, unittest.TestSuite): 690 raise ValueError('Custom test function {0} must return a ' 691 'unittest.TestSuite, not {1}'.format(obj.__name__, type(suite))) 692 filtered.append(suite) 693 694 return filtered
695
696 -def unit(klass):
697 """ 698 A class decorator, used to label unit test cases. 699 700 @param klass: a C{unittest.TestCase} class type 701 @type klass: type 702 """ 703 if not isinstance(klass, type): 704 raise TypeError("Can't apply class decorator on {0}".format(type(klass))) 705 706 setattr(klass, Attributes.UNIT, True) 707 return klass
708
709 -def functional(klass):
710 """ 711 A class decorator, used to label functional test cases. 712 713 @param klass: a C{unittest.TestCase} class type 714 @type klass: type 715 """ 716 if not isinstance(klass, type): 717 raise TypeError("Can't apply class decorator on {0}".format(type(klass))) 718 719 setattr(klass, Attributes.FUNCTIONAL, True) 720 return klass
721
722 -def regression(klass):
723 """ 724 A class decorator, used to label regression test cases. 725 726 @param klass: a C{unittest.TestCase} class type 727 @type klass: type 728 """ 729 if not isinstance(klass, type): 730 raise TypeError("Can't apply class decorator on {0}".format(type(klass))) 731 732 setattr(klass, Attributes.REGRESSION, True) 733 return klass
734
735 -def custom(function):
736 """ 737 A function decorator, used to mark functions which build custom (dynamic) 738 test suites when called. 739 740 @param function: a callable object, which returns a dynamically compiled 741 C{unittest.TestSuite} 742 @type function: callable 743 """ 744 if isinstance(function, type): 745 raise TypeError("Can't apply function decorator on a class") 746 elif not hasattr(function, '__call__'): 747 raise TypeError("Can't apply function decorator on non-callable {0}".format(type(function))) 748 749 setattr(function, Attributes.CUSTOM, True) 750 return function
751
752 -def skip(reason, condition=None):
753 """ 754 Mark a test case or method for skipping. 755 756 @param reason: message 757 @type reason: str 758 @param condition: skip only if the specified condition is True 759 @type condition: bool/expression 760 """ 761 if isinstance(reason, types.FunctionType): 762 raise TypeError('skip: no reason specified') 763 764 if condition is None: 765 return unittest.skip(reason) 766 else: 767 return unittest.skipIf(condition, reason)
768
769 -class Console(object):
770 """ 771 Build and run all tests of the specified namespace and kind. 772 773 @param namespace: a dotted name, which specifies the test module 774 (see L{csb.test.AbstractTestBuilder.loadTests}) 775 @type namespace: str 776 @param builder: test builder to use 777 @type builder: any L{csb.test.AbstractTestBuilder} subclass 778 @param verbosity: verbosity level for C{unittest.TestRunner} 779 @type verbosity: int 780 @param update: if True, refresh all pickles in csb/test/data 781 @type update: bool 782 @param generated_data: where to cache generated test files (directory) 783 @type generated_data: str 784 """ 785 786 BUILDERS = {'unit': UnitTestBuilder, 'functional': FunctionalTestBuilder, 787 'custom': CustomTestBuilder, 'any': AnyTestBuilder, 788 'regression': RegressionTestBuilder} 789 790
791 - def __init__(self, namespace=('__main__',), builder=AnyTestBuilder, verbosity=1, 792 update=False, generated_data=Config.GENERATED_DATA, argv=None):
793 794 if not argv: 795 argv = sys.argv 796 797 self._namespace = None 798 self._builder = None 799 self._verbosity = 1 800 self._update = False 801 self._gendata = str(generated_data) 802 self._program = os.path.basename(argv[0]) 803 804 self.namespace = namespace 805 self.builder = builder 806 self.verbosity = verbosity 807 self.update = update 808 self.generated_data = generated_data 809 810 self.parseArguments(argv[1:]) 811 self.run()
812 813 @property
814 - def namespace(self):
815 return self._namespace
816 @namespace.setter
817 - def namespace(self, value):
818 if csb.core.iterable(value): 819 self._namespace = list(value) 820 else: 821 self._namespace = [value]
822 823 @property
824 - def builder(self):
825 return self._builder
826 @builder.setter
827 - def builder(self, value):
828 self._builder = value
829 830 @property
831 - def verbosity(self):
832 return self._verbosity
833 @verbosity.setter
834 - def verbosity(self, value):
835 self._verbosity = value
836 837 @property
838 - def builders(self):
839 return ', '.join(Console.BUILDERS)
840 841 @property
842 - def program(self):
843 return self._program
844 845 @property
846 - def update(self):
847 return self._update
848 @update.setter
849 - def update(self, value):
850 self._update = bool(value)
851 852 @property
853 - def generated_data(self):
854 return self._gendata
855 @generated_data.setter
856 - def generated_data(self, value):
857 self._gendata = os.path.abspath(value)
858
859 - def run(self):
860 861 Config.setDefaultGeneratedDataRoot(self.generated_data) 862 863 if self.update: 864 Config().updateDataFiles() 865 else: 866 Config().ensureDataConsistency() 867 868 builder = self.builder() 869 suite = builder.loadMultipleTests(self.namespace) 870 871 runner = unittest.TextTestRunner(verbosity=self.verbosity) 872 runner.run(suite)
873
874 - def parseArguments(self, argv):
875 876 parser = argparse.ArgumentParser(prog=self.program, description="CSB Test Runner Console.") 877 878 parser.add_argument("-t", "--type", type=str, default="any", choices=list(Console.BUILDERS), 879 help="Type of tests to load from each namespace (default=any)") 880 parser.add_argument("-v", "--verbosity", type=int, default=1, 881 help="Verbosity level passed to unittest.TextTestRunner (default=1).") 882 parser.add_argument("-u", "--update-files", default=False, action="store_true", 883 help="Force update of the test pickles in " + Config.GENERATED_DATA) 884 parser.add_argument("-g", "--generated-resources", type=str, default=Config.GENERATED_DATA, 885 help="Generate, store and load additional test resources in this directory" 886 " (default=" + Config.GENERATED_DATA + ")") 887 888 parser.add_argument("namespaces", nargs='*', 889 help="""An optional list of CSB test dotted namespaces, from which to 890 load tests. '__main__' and '.' are interpreted as the 891 current module. If a namespace ends with an asterisk 892 '.*', all sub-packages will be scanned as well. 893 894 Examples: 895 "csb.test.cases.bio.*" 896 "csb.test.cases.bio.io" "csb.test.cases.bio.utils" 897 ".")""") 898 899 args = parser.parse_args(argv) 900 901 self.builder = Console.BUILDERS[args.type] 902 self.verbosity = args.verbosity 903 self.update = args.update_files 904 self.generated_data = args.generated_resources 905 906 if args.namespaces: 907 self.namespace = args.namespaces
908 909 910 if __name__ == '__main__': 911 912 Console() 913