Simple class that stores security context information in the web request.
Projects should subclass this class if they wish to enhance the request context or provide additional information in their specific WSGI pipeline.
Bases: object
Helper class to represent useful information about a request context.
Stores information about the security context under which the user accesses the system, as well as additional request information.
Exceptions common to OpenStack projects
Bases: exceptions.Exception
Error resulting from a client sending bad input to a server
Bases: keystone.openstack.common.exception.Error
Bases: exceptions.Exception
Error resulting from a client connecting to a server
Bases: exceptions.Exception
Bases: keystone.openstack.common.exception.OpenstackException
Bases: keystone.openstack.common.exception.OpenstackException
Bases: exceptions.Exception
Base Exception class.
To correctly use this class, inherit from it and define a ‘msg_fmt’ property. That message will get printf’d with the keyword arguments provided to the constructor.
Bases: keystone.openstack.common.exception.Error
Exception related utilities.
Bases: object
Save current exception, run some code and then re-raise.
In some cases the exception context can be cleared, resulting in None being attempted to be re-raised after an exception handler is run. This can happen when eventlet switches greenthreads or when running an exception handler, code raises and catches an exception. In both cases the exception context will be cleared.
To work around this, we save the exception state, run handler code, and then re-raise the original exception. If another exception occurs, the saved exception is logged and the new exception is re-raised.
In some cases the caller may not want to re-raise the exception, and for those circumstances this context provides a reraise flag that can be used to suppress the exception. For example:
except Exception:
with save_and_reraise_exception() as ctxt:
decide_if_need_reraise()
if not should_be_reraised:
ctxt.reraise = False
Delete a file, but ignore file not found error.
Parameters: |
|
---|
Create a directory (and any ancestor directories required)
Parameters: | path – Directory to create |
---|
Open file
see built-in file() documentation for more details
Note: The reason this is kept in a separate module is to easily be able to provide a stub module that doesn’t alter system state at all (for unit tests)
Read from a file if it has been modified.
Parameters: | force_reload – Whether to reload the file. |
---|---|
Returns: | A tuple with a boolean specifying if the data is fresh or not. |
Protect code that wants to operate on PATH atomically. Any exception will cause PATH to be removed.
Parameters: |
|
---|
Create temporary file or use existing file.
This util is needed for creating temporary file with specified content, suffix and prefix. If path is not None, it will be used for writing content. If the path doesn’t exist it’ll be created.
Parameters: |
|
---|
For example: it can be used in database tests for creating configuration files.
gettext for openstack-common modules.
Usual usage in an openstack.common module:
from keystone.openstack.common.gettextutils import _
Bases: unicode
A Message object is a unicode object that can be translated.
Translation of Message is done explicitly using the translate() method. For all non-translation intents and purposes, a Message is simply unicode, and can be treated as such.
Bases: logging.handlers.MemoryHandler
Handler that translates records before logging them.
The TranslationHandler takes a locale and a target logging.Handler object to forward LogRecord objects to after translating them. This handler depends on Message objects being logged, instead of regular strings.
The handler can be configured declaratively in the logging.conf as follows:
[handlers] keys = translatedlog, translator
[handler_translatedlog] class = handlers.WatchedFileHandler args = (‘/var/log/api-localized.log’,) formatter = context
[handler_translator] class = openstack.common.log.TranslationHandler target = translatedlog args = (‘zh_CN’,)
If the specified locale is not available in the system, the handler will log in the default locale.
Convenience function for configuring _() to use lazy gettext
Call this at the start of execution to enable the gettextutils._ function to use lazy gettext functionality. This is useful if your project is importing _ directly instead of using the gettextutils.install() way of importing the _ function.
Lists the available languages for the given translation domain.
Parameters: | domain – the domain to get languages for |
---|
Install a _() function using the given translation domain.
Given a translation domain, install a _() function using gettext’s install() function.
The main difference from gettext.install() is that we allow overriding the default localedir (e.g. /usr/share/locale) using a translation-domain-specific environment variable (e.g. NOVA_LOCALEDIR).
Parameters: |
|
---|
Gets the translated unicode representation of the given object.
If the object is not translatable it is returned as-is. If the locale is None the object is translated to the system locale.
Parameters: |
|
---|---|
Returns: | the translated object in unicode, or the original object if it could not be translated |
Import related utilities and helper functions.
Returns a class from a string including module and class.
Import a class and return an instance of it.
Tries to import object from default namespace.
Imports a class and return an instance of it, first by trying to find the class in a default namespace, then failing back to a full path if not found in the default namespace.
JSON related utilities.
This module provides a few things:
1) A handy function for getting an object down to something that can be JSON serialized. See to_primitive().
2) Wrappers around loads() and dumps(). The dumps() wrapper will automatically use to_primitive() for you if needed.
3) This sets up anyjson to use the loads() and dumps() wrappers if anyjson is available.
Convert a complex object into primitives.
Handy for JSON serialization. We can optionally handle instances, but since this is a recursive function, we could have cyclical data structures.
To handle cyclical data structures we could track the actual objects visited in a set, but not all objects are hashable. Instead we just track the depth of the object inspections and don’t go too deep.
Therefore, convert_instances=True is lossy ... be aware.
Local storage of variables using weak references
alias of _PosixLock
Context based lock
This function yields a threading.Semaphore instance (if we don’t use eventlet.monkey_patch(), else semaphore.Semaphore) unless external is True, in which case, it’ll yield an InterProcessLock instance.
Parameters: |
|
---|
Create a dir for locks and pass it to command from arguments
If you run this: python -m openstack.common.lockutils python setup.py testr <etc>
a temporary directory will be created for all your locks and passed to all your tests in an environment variable. The temporary dir will be deleted afterwards and the return value will be preserved.
Synchronization decorator.
Decorating a method like so:
@synchronized('mylock')
def foo(self, *args):
...
ensures that only one thread will execute the foo method at a time.
Different methods can share the same lock:
@synchronized('mylock')
def foo(self, *args):
...
@synchronized('mylock')
def bar(self, *args):
...
This way only one of either foo or bar can be executing at a time.
Partial object generator for the synchronization decorator.
Redefine @synchronized in each project like so:
(in nova/utils.py)
from nova.openstack.common import lockutils
synchronized = lockutils.synchronized_with_prefix('nova-')
(in nova/foo.py)
from nova import utils
@utils.synchronized('mylock')
def bar(self, *args):
...
The lock_file_prefix argument is used to provide lock files on disk with a meaningful prefix.
Openstack logging handler.
This module adds to logging functionality by adding the option to specify a context object when calling the various log methods. If the context object is not specified, default formatting is used. Additionally, an instance uuid may be passed as part of the log message, which is intended to make it easier for admins to find messages related to a specific instance.
It also allows setting of formatting information through conf.
Bases: logging.LoggerAdapter
Bases: logging.StreamHandler
Bases: keystone.openstack.common.log.BaseLoggerAdapter
Call this method when a deprecated feature is used.
If the system is configured for fatal deprecations then the message is logged at the ‘critical’ level and DeprecatedConfig will be raised.
Otherwise, the message will be logged (once) at the ‘warn’ level.
Raises: | DeprecatedConfig if the system is configured for fatal deprecations. |
---|
Delegate a warning call to the underlying logger, after adding contextual information from this adapter instance.
Bases: logging.Formatter
A context.RequestContext aware formatter configured through flags.
The flags used to set format strings are: logging_context_format_string and logging_default_format_string. You can also specify logging_debug_format_suffix to append extra formatting if the log level is debug.
For information about what variables are available for the formatter see: http://docs.python.org/library/logging.html#formatter
If available, uses the context value stored in TLS - local.store.context
Bases: exceptions.Exception
Bases: logging.Formatter
Bases: exceptions.Exception
Bases: logging.handlers.SysLogHandler
Bases: object
A thin wrapper that responds to write and logs.
Returns lazy logger.
Creates a pass-through logger that does not create the real logger until it is really needed and delegates all calls to the real logger once it is created.
Replace password with ‘secret’ in message.
Parameters: |
|
---|---|
Returns: | The unicode value of message with the password fields masked. |
For example:
>>> mask_password("'adminPass' : 'aaaaa'")
"'adminPass' : '***'"
>>> mask_password("'admin_pass' : 'aaaaa'")
"'admin_pass' : '***'"
>>> mask_password('"password" : "aaaaa"')
'"password" : "***"'
>>> mask_password("'original_password' : 'aaaaa'")
"'original_password' : '***'"
>>> mask_password("u'original_password' : u'aaaaa'")
"u'original_password' : u'***'"
Bases: keystone.openstack.common.loopingcall.LoopingCallBase
A looping call which sleeps until the next known event.
The function called should return how long to sleep for before being called again.
Bases: keystone.openstack.common.loopingcall.LoopingCallBase
A fixed interval looping call.
alias of FixedIntervalLoopingCall
Bases: object
Bases: exceptions.Exception
Exception to break out and stop a LoopingCall.
The poll-function passed to LoopingCall can raise this exception to break out of the loop normally. This is somewhat analogous to StopIteration.
An optional return-value can be included as the argument to the exception; this return-value will be returned by LoopingCall.wait()
Network-related utilities and helper functions.
Interpret a string as a host:port pair.
An IPv6 address MUST be escaped if accompanied by a port, because otherwise ambiguity ensues: 2001:db8:85a3::8a2e:370:7334 means both [2001:db8:85a3::8a2e:370:7334] and [2001:db8:85a3::8a2e:370]:7334.
>>> parse_host_port('server01:80')
('server01', 80)
>>> parse_host_port('server01')
('server01', None)
>>> parse_host_port('server01', default_port=1234)
('server01', 1234)
>>> parse_host_port('[::1]:80')
('::1', 80)
>>> parse_host_port('[::1]')
('::1', None)
>>> parse_host_port('[::1]', default_port=1234)
('::1', 1234)
>>> parse_host_port('2001:db8:85a3::8a2e:370:7334', default_port=1234)
('2001:db8:85a3::8a2e:370:7334', 1234)
Common Policy Engine Implementation
Policies can be expressed in one of two forms: A list of lists, or a string written in the new policy language.
In the list-of-lists representation, each check inside the innermost list is combined as with an “and” conjunction–for that check to pass, all the specified checks must pass. These innermost lists are then combined as with an “or” conjunction. This is the original way of expressing policies, but there now exists a new way: the policy language.
In the policy language, each check is specified the same way as in the list-of-lists representation: a simple “a:b” pair that is matched to the correct code to perform that check. However, conjunction operators are available, allowing for more expressiveness in crafting policies.
As an example, take the following rule, expressed in the list-of-lists representation:
[["role:admin"], ["project_id:%(project_id)s", "role:projectadmin"]]
In the policy language, this becomes:
role:admin or (project_id:%(project_id)s and role:projectadmin)
The policy language also has the “not” operator, allowing a richer policy rule:
project_id:%(project_id)s and not role:dunce
Finally, two special policy checks should be mentioned; the policy check “@” will always accept an access, and the policy check ”!” will always reject an access. (Note that if a rule is either the empty list (“[]”) or the empty string, this is equivalent to the “@” policy check.) Of these, the ”!” policy check is probably the most useful, as it allows particular rules to be explicitly disabled.
Bases: keystone.openstack.common.policy.BaseCheck
Implements the “and” logical operator.
A policy check that requires that a list of other checks all return True.
Bases: object
Abstract base class for Check classes.
Bases: keystone.openstack.common.policy.BaseCheck
A base class to allow for user-defined policy checks.
Bases: object
Responsible for loading and enforcing rules.
Parameters: |
|
---|
Checks authorization of a rule against the target and credentials.
Parameters: |
|
---|---|
Returns: | Returns False if the policy does not allow the action and exc is not provided; otherwise, returns a value that evaluates to True. Note: for rules using the “case” expression, this True value will be the specified string from the expression. |
Bases: keystone.openstack.common.policy.BaseCheck
A policy check that always returns False (disallow).
Bases: keystone.openstack.common.policy.BaseCheck
Implements the “not” logical operator.
A policy check that inverts the result of another policy check.
Bases: keystone.openstack.common.policy.BaseCheck
Implements the “or” operator.
A policy check that requires that at least one of a list of other checks returns True.
Bases: object
Implement the core of parsing the policy language.
Uses a greedy reduction algorithm to reduce a sequence of tokens into a single terminal, the value of which will be the root of the Check tree.
Note: error reporting is rather lacking. The best we can get with this parser formulation is an overall “parse failed” error. Fortunately, the policy language is simple enough that this shouldn’t be that big a problem.
Perform a greedy reduction of the token stream.
If a reducer method matches, it will be executed, then the reduce() method will be called recursively to search for any more possible reductions.
Bases: type
Metaclass for the ParseState class.
Facilitates identifying reduction methods.
Bases: exceptions.Exception
Bases: dict
A store for rules. Handles the default_rule setting directly.
Bases: keystone.openstack.common.policy.BaseCheck
A policy check that always returns True (allow).
Parses a policy rule into a tree of Check objects.
Decorator for reduction methods.
Arguments are a sequence of tokens, in order, which should trigger running this reduction method.
Register a function or Check class as a policy check.
Parameters: |
|
---|
System-level utilities and helper functions.
Bases: exceptions.Exception
Bases: exceptions.Exception
Bases: exceptions.Exception
Bases: exceptions.Exception
Helper method to shell out and execute a command through subprocess.
Allows optional retry.
Parameters: |
|
---|---|
Returns: | (stdout, stderr) from process execution |
Raises: | UnknownArgumentError on receiving unknown arguments |
Raises: |
A wrapper around execute() to more easily handle warnings and errors.
Returns an (out, err) tuple of strings containing the output of the command’s stdout and stderr. If ‘err’ is not empty then the command can be considered to have failed.
Generic Node base class for all workers that run on hosts.
Bases: object
Launch one or more services and wait for them to complete.
Bases: object
Service object for binaries running on hosts.
Bases: object
System-level utilities and helper functions.
Interpret a string as a boolean.
A case-insensitive match is performed such that strings matching ‘t’, ‘true’, ‘on’, ‘y’, ‘yes’, or ‘1’ are considered True and, when strict=False, anything else returns the value specified by ‘default’.
Useful for JSON-decoded stuff and config file parsing.
If strict=True, unrecognized values, including None, will raise a ValueError which is useful when parsing values passed in from an API call. Strings yielding False are ‘f’, ‘false’, ‘off’, ‘n’, ‘no’, or ‘0’.
Interpret a string as a boolean and return either 1 or 0.
Any string value in:
(‘True’, ‘true’, ‘On’, ‘on’, ‘1’)
is interpreted as a boolean True.
Useful for JSON-decoded stuff and config file parsing
Decodes incoming str using incoming if they’re not already unicode.
Parameters: |
|
---|---|
Returns: | text or a unicode incoming encoded representation of it. |
Raises TypeError: | |
If text is not an instance of str |
Encodes incoming str/unicode using encoding.
If incoming is not specified, text is expected to be encoded with current python’s default encoding. (sys.getdefaultencoding)
Parameters: |
|
---|---|
Returns: | text or a bytestring encoding encoded representation of it. |
Raises TypeError: | |
If text is not an instance of str |
Converts a string into an integer of bytes.
Looks at the last characters of the text to determine what conversion is needed to turn the input text into a byte number. Supports “B, K(B), M(B), G(B), and T(B)”. (case insensitive)
Parameters: |
|
---|
Normalize string.
Convert to lowercase, remove non-word characters, and convert spaces to hyphens.
Inspired by Django’s slugify filter.
Parameters: |
|
---|---|
Returns: | slugified unicode representation of value |
Raises TypeError: | |
If text is not an instance of str |
Common utilities used in testing
Bases: object
Wrapper around a greenthread, that holds a reference to the ThreadGroup. The Thread will notify the ThreadGroup when it has done so it can be removed from the threads list.
Bases: object
The point of the ThreadGroup classis to:
Time related utilities and helper functions.
Advance overridden time using a datetime.timedelta.
Advance overridden time by seconds.
Return the difference between two timing objects.
Compute the difference in seconds between two date, time, or datetime objects (as a float, to microsecond resolution).
Return True if after is newer than seconds.
Return True if before is older than seconds.
Determines if time is going to happen in the next window seconds.
Params dt: | the time |
---|---|
Params window: | minimum seconds to remain to consider the time not soon |
Returns: | True if expiration is within the given duration |
Returns a iso8601 formated date from timestamp.
Stringify time in ISO 8601 format.
Make an rpc-safe datetime with microseconds.
Note: tzinfo is stripped, but not required for relative times.
Normalize time in arbitrary timezone to UTC naive object.
Parse time from ISO 8601 format.
Turn a formatted time back into a datetime.
Overrides utils.utcnow.
Make it return a constant time or a list thereof, one at a time.
Parameters: | override_time – datetime instance or list thereof. If not given, defaults to the current UTC time. |
---|
Returns formatted utcnow.
UUID related utilities and helper functions.
Helpers for comparing version strings.
Bases: object
A decorator to mark callables as deprecated.
deprecated logs a deprecation message when the callable it decorates is used. The message will include the release where the callable was deprecated, the release where is may be removed and possibly an optional replacement.
Examples:
>>> @deprecated(as_of=deprecated.ICEHOUSE)
... def a(): pass
>>> @deprecated(as_of=deprecated.ICEHOUSE, in_favor_of='f()')
... def b(): pass
>>> @deprecated(as_of=deprecated.ICEHOUSE, remove_in=+1)
... def c(): pass
Determine whether requested_version is satisfied by current_version; in other words, current_version is >= requested_version.
Parameters: |
|
---|---|
Returns: | True if compatible, False if not |