misc

Miscellaneous modules for translate - including modules for backward compatibility with pre-2.3 versions of Python

autoencode

Supports a hybrid Unicode string that knows which encoding is preferable, and uses this when converting to a string.

contextlib

Utilities for with-statement contexts. See PEP 343.

translate.misc.contextlib.contextmanager(func)

@contextmanager decorator.

Typical usage:

@contextmanager
def some_generator(<arguments>):
    <setup>
    try:
        yield <value>
    finally:
        <cleanup>

This makes this:

with some_generator(<arguments>) as <variable>:
    <body>

equivalent to this:

<setup>
try:
    <variable> = <value>
    <body>
finally:
    <cleanup>
translate.misc.contextlib.nested(*args, **kwds)

Support multiple context managers in a single with-statement.

Code like this:

with nested(A, B, C) as (X, Y, Z):
    <body>

is equivalent to this:

with A as X:
    with B as Y:
        with C as Z:
            <body>
class translate.misc.contextlib.closing(thing)

Context to automatically close something at the end of a block.

Code like this:

with closing(<module>.open(<arguments>)) as f:
    <block>

is equivalent to this:

f = <module>.open(<arguments>)
try:
    <block>
finally:
    f.close()

context

translate.misc.context.with_(mgr, body)

A function to mimic the with statement introduced in Python 2.5

The code below was taken from http://www.python.org/dev/peps/pep-0343/

dictutils

Implements a case-insensitive (on keys) dictionary and order-sensitive dictionary

translate.misc.dictutils.generalupper(str)

this uses the object’s upper method - works with string and unicode

class translate.misc.dictutils.ordereddict(*args)

a dictionary which remembers its keys in the order in which they were given

clear() → None. Remove all items from D.
copy() → a shallow copy of D
static fromkeys(S[, v]) → New dict with keys from S and values equal to v.

v defaults to None.

get(k[, d]) → D[k] if k in D, else d. d defaults to None.
has_key(k) → True if D has a key k, else False
items() → list of D's (key, value) pairs, as 2-tuples
iteritems() → an iterator over the (key, value) items of D
iterkeys() → an iterator over the keys of D
itervalues() → an iterator over the values of D
keys() → list of D's keys
pop(key)

remove entry from dict and internal list

popitem() → (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D
update(E) → None.

Update D from E: for k in E.keys(): D[k] = E[k]

values() → list of D's values
viewitems() → a set-like object providing a view on D's items
viewkeys() → a set-like object providing a view on D's keys
viewvalues() → an object providing a view on D's values

diff_match_patch

Diff Match and Patch

Copyright 2006 Google Inc. http://code.google.com/p/google-diff-match-patch/

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

class translate.misc.diff_match_patch.diff_match_patch

Class containing the diff, match and patch methods.

Also contains the behaviour settings.

diff_charsToLines(diffs, lineArray)

Rehydrate the text in a diff from a string of line hashes to real lines of text.

Parameters:
  • diffs – Array of diff tuples.
  • lineArray – Array of unique strings.
diff_cleanupEfficiency(diffs)

Reduce the number of edits by eliminating operationally trivial equalities.

Parameters:diffs – Array of diff tuples.
diff_cleanupMerge(diffs)

Reorder and merge like edit sections. Merge equalities. Any edit section can move as long as it doesn’t cross an equality.

Parameters:diffs – Array of diff tuples.
diff_cleanupSemantic(diffs)

Reduce the number of edits by eliminating semantically trivial equalities.

Parameters:diffs – Array of diff tuples.
diff_cleanupSemanticLossless(diffs)

Look for single edits surrounded on both sides by equalities which can be shifted sideways to align the edit to a word boundary. e.g: The c<ins>at c</ins>ame. -> The <ins>cat </ins>came.

Parameters:diffs – Array of diff tuples.
diff_commonPrefix(text1, text2)

Determine the common prefix of two strings.

Parameters:
  • text1 – First string.
  • text2 – Second string.
Returns:

The number of characters common to the start of each string.

diff_commonSuffix(text1, text2)

Determine the common suffix of two strings.

Parameters:
  • text1 – First string.
  • text2 – Second string.
Returns:

The number of characters common to the end of each string.

diff_compute(text1, text2, checklines)

Find the differences between two texts. Assumes that the texts do not have any common prefix or suffix.

Parameters:
  • text1 – Old string to be diffed.
  • text2 – New string to be diffed.
  • checklines – Speedup flag. If false, then don’t run a line-level diff first to identify the changed areas. If True, then run a faster, slightly less optimal diff.
Returns:

Array of changes.

diff_fromDelta(text1, delta)

Given the original text1, and an encoded string which describes the operations required to transform text1 into text2, compute the full diff.

Parameters:
  • text1 – Source string for the diff.
  • delta – Delta text.
Returns:

Array of diff tuples.

Raises ValueError:
 

If invalid input.

diff_halfMatch(text1, text2)

Do the two texts share a substring which is at least half the length of the longer text?

Parameters:
  • text1 – First string.
  • text2 – Second string.
Returns:

Five element Array, containing the prefix of text1, the suffix of text1, the prefix of text2, the suffix of text2 and the common middle. Or None if there was no match.

diff_levenshtein(diffs)

Compute the Levenshtein distance; the number of inserted, deleted or substituted characters.

Parameters:diffs – Array of diff tuples.
Returns:Number of changes.
diff_linesToChars(text1, text2)

Split two texts into an array of strings. Reduce the texts to a string of hashes where each Unicode character represents one line.

Parameters:
  • text1 – First string.
  • text2 – Second string.
Returns:

Three element tuple, containing the encoded text1, the encoded text2 and the array of unique strings. The zeroth element of the array of unique strings is intentionally blank.

diff_main(text1, text2, checklines=True)

Find the differences between two texts. Simplifies the problem by stripping any common prefix or suffix off the texts before diffing.

Parameters:
  • text1 – Old string to be diffed.
  • text2 – New string to be diffed.
  • checklines – Optional speedup flag. If present and false, then don’t run a line-level diff first to identify the changed areas. Defaults to True, which does a faster, slightly less optimal diff.
Returns:

Array of changes.

diff_map(text1, text2)

Explore the intersection points between the two texts.

Parameters:
  • text1 – Old string to be diffed.
  • text2 – New string to be diffed.
Returns:

Array of diff tuples or None if no diff available.

diff_path1(v_map, text1, text2)

Work from the middle back to the start to determine the path.

Parameters:
  • v_map – Array of paths.
  • text1 – Old string fragment to be diffed.
  • text2 – New string fragment to be diffed.
Returns:

Array of diff tuples.

diff_path2(v_map, text1, text2)

Work from the middle back to the end to determine the path.

Parameters:
  • v_map – Array of paths.
  • text1 – Old string fragment to be diffed.
  • text2 – New string fragment to be diffed.
Returns:

Array of diff tuples.

diff_prettyHtml(diffs)

Convert a diff array into a pretty HTML report.

Parameters:diffs – Array of diff tuples.
Returns:HTML representation.
diff_text1(diffs)

Compute and return the source text (all equalities and deletions).

Parameters:diffs – Array of diff tuples.
Returns:Source text.
diff_text2(diffs)

Compute and return the destination text (all equalities and insertions).

Parameters:diffs – Array of diff tuples.
Returns:Destination text.
diff_toDelta(diffs)

Crush the diff into an encoded string which describes the operations required to transform text1 into text2. E.g. =3 -2 +ing -> Keep 3 chars, delete 2 chars, insert ‘ing’. Operations are tab-separated. Inserted text is escaped using %xx notation.

Parameters:diffs – Array of diff tuples.
Returns:Delta text.
diff_xIndex(diffs, loc)

loc is a location in text1, compute and return the equivalent location in text2. e.g. “The cat” vs “The big cat”, 1->1, 5->8

Parameters:
  • diffs – Array of diff tuples.
  • loc – Location within text1.
Returns:

Location within text2.

match_alphabet(pattern)

Initialise the alphabet for the Bitap algorithm.

Parameters:pattern – The text to encode.
Returns:Hash of character locations.
match_bitap(text, pattern, loc)

Locate the best instance of ‘pattern’ in ‘text’ near ‘loc’ using the Bitap algorithm.

Parameters:
  • text – The text to search.
  • pattern – The pattern to search for.
  • loc – The location to search around.
Returns:

Best match index or -1.

match_main(text, pattern, loc)

Locate the best instance of ‘pattern’ in ‘text’ near ‘loc’.

Parameters:
  • text – The text to search.
  • pattern – The pattern to search for.
  • loc – The location to search around.
Returns:

Best match index or -1.

patch_addContext(patch, text)

Increase the context until it is unique, but don’t let the pattern expand beyond Match_MaxBits.

Parameters:
  • patch – The patch to grow.
  • text – Source text.
patch_addPadding(patches)

Add some padding on text start and end so that edges can match something. Intended to be called only from within patch_apply.

Parameters:patches – Array of patch objects.
Returns:The padding string added to each side.
patch_apply(patches, text)

Merge a set of patches onto the text. Return a patched text, as well as a list of true/false values indicating which patches were applied.

Parameters:
  • patches – Array of patch objects.
  • text – Old text.
Returns:

Two element Array, containing the new text and an array of boolean values.

patch_deepCopy(patches)

Given an array of patches, return another array that is identical.

Parameters:patches – Array of patch objects.
Returns:Array of patch objects.
patch_fromText(textline)

Parse a textual representation of patches and return a list of patch objects.

Parameters:textline – Text representation of patches.
Returns:Array of patch objects.
Raises ValueError:
 If invalid input.
patch_make(a, b=None, c=None)

Compute a list of patches to turn text1 into text2. Use diffs if provided, otherwise compute it ourselves. There are four ways to call this function, depending on what data is available to the caller: Method 1: a = text1, b = text2 Method 2: a = diffs Method 3 (optimal): a = text1, b = diffs Method 4 (deprecated, use method 3): a = text1, b = text2, c = diffs

Parameters:
  • a – text1 (methods 1,3,4) or Array of diff tuples for text1 to text2 (method 2).
  • b – text2 (methods 1,4) or Array of diff tuples for text1 to text2 (method 3) or undefined (method 2).
  • c – Array of diff tuples for text1 to text2 (method 4) or undefined (methods 1,2,3).
Returns:

Array of patch objects.

patch_splitMax(patches)

Look through the patches and break up any which are longer than the maximum limit of the match algorithm.

Parameters:patches – Array of patch objects.
patch_toText(patches)

Take a list of patches and return a textual representation.

Parameters:patches – Array of patch objects.
Returns:Text representation of patches.
class translate.misc.diff_match_patch.patch_obj

Class representing one patch operation.

file_discovery

translate.misc.file_discovery.get_abs_data_filename(path_parts, basedirs=None)

Get the absolute path to the given file- or directory name in the current running application’s data directory.

Parameters:path_parts (list) – The path parts that can be joined by os.path.join().

hash

This module contains some temporary glue to make us work with md5 hashes on old and new versions of Python. The function md5_f() wraps whatever is available.

ini

Access and/or modify INI files

  • Compatiable with ConfigParser
  • Preserves order of sections & options
  • Preserves comments/blank lines/etc
  • More convenient access to data

Example:

>>> from StringIO import StringIO
>>> sio = StringIO('''# configure foo-application
... [foo]
... bar1 = qualia
... bar2 = 1977
... [foo-ext]
... special = 1''')
>>> cfg = INIConfig(sio)
>>> print cfg.foo.bar1
qualia
>>> print cfg['foo-ext'].special
1
>>> cfg.foo.newopt = 'hi!'
>>> print cfg
# configure foo-application
[foo]
bar1 = qualia
bar2 = 1977
newopt = hi!
[foo-ext]
special = 1
translate.misc.ini.readline_iterator(f)

iterate over a file by only using the file object’s readline method

lru

class translate.misc.lru.LRUCachingDict(maxsize, cullsize=2, peakmult=10, aggressive_gc=True, *args, **kwargs)

Caching dictionary like object that discards the least recently used objects when number of cached items exceeds maxsize.

cullsize is the fraction of items that will be discarded when maxsize is reached.

cull()

free memory by deleting old items from cache

itervaluerefs()

Return an iterator that yields the weak references to the values.

The references are not guaranteed to be ‘live’ at the time they are used, so the result of calling the references needs to be checked before being used. This can be used to avoid creating references that will cause the garbage collector to keep the values around longer than needed.

valuerefs()

Return a list of weak references to the values.

The references are not guaranteed to be ‘live’ at the time they are used, so the result of calling the references needs to be checked before being used. This can be used to avoid creating references that will cause the garbage collector to keep the values around longer than needed.

multistring

Supports a hybrid Unicode string that can also have a list of alternate strings in the strings attribute

optrecurse

class translate.misc.optrecurse.RecursiveOptionParser(formats, usetemplates=False, allowmissingtemplate=False, description=None)

A specialized Option Parser for recursing through directories.

add_option(Option)

add_option(opt_str, ..., kwarg=val, ...)

check_values(values : Values, args : [string])

-> (values : Values, args : [string])

Check that the supplied option values and leftover arguments are valid. Returns the option values and leftover arguments (possibly adjusted, possibly completely new – whatever you like). Default implementation just returns the passed-in values; subclasses may override as desired.

checkoutputsubdir(options, subdir)

Checks to see if subdir under options.output needs to be created, creates if neccessary.

define_option(option)

Defines the given option, replacing an existing one of the same short name if neccessary...

destroy()

Declare that you are done with this OptionParser. This cleans up reference cycles so the OptionParser (and all objects referenced by it) can be garbage-collected promptly. After calling destroy(), the OptionParser is unusable.

disable_interspersed_args()

Set parsing to stop on the first non-option. Use this if you have a command processor which runs another command that has options of its own and you want to make sure these options don’t get confused.

enable_interspersed_args()

Set parsing to not stop on the first non-option, allowing interspersing switches with command arguments. This is the default behavior. See also disable_interspersed_args() and the class documentation description of the attribute allow_interspersed_args.

error(msg : string)

Print a usage message incorporating ‘msg’ to stderr and exit. If you override this in a subclass, it should not return – it should either exit or raise an exception.

finalizetempoutputfile(options, outputfile, fulloutputpath)

Write the temp outputfile to its final destination.

format_manpage()

returns a formatted manpage

getformathelp(formats)

Make a nice help string for describing formats...

getfullinputpath(options, inputpath)

Gets the absolute path to an input file.

getfulloutputpath(options, outputpath)

Gets the absolute path to an output file.

getfulltemplatepath(options, templatepath)

Gets the absolute path to a template file.

getoutputname(options, inputname, outputformat)

Gets an output filename based on the input filename.

getoutputoptions(options, inputpath, templatepath)

Works out which output format and processor method to use...

getpassthroughoptions(options)

Get the options required to pass to the filtermethod...

gettemplatename(options, inputname)

Gets an output filename based on the input filename.

getusageman(option)

returns the usage string for the given option

getusagestring(option)

returns the usage string for the given option

initprogressbar(allfiles, options)

Sets up a progress bar appropriate to the options and files.

isexcluded(options, inputpath)

Checks if this path has been excluded.

isrecursive(fileoption, filepurpose='input')

Checks if fileoption is a recursive file.

isvalidinputname(options, inputname)

Checks if this is a valid input filename.

mkdir(parent, subdir)

Makes a subdirectory (recursively if neccessary).

openinputfile(options, fullinputpath)

Opens the input file.

openoutputfile(options, fulloutputpath)

Opens the output file.

opentemplatefile(options, fulltemplatepath)

Opens the template file (if required).

opentempoutputfile(options, fulloutputpath)

Opens a temporary output file.

parse_args(args=None, values=None)

Parses the command line options, handling implicit input/output args.

print_help(file : file = stdout)

Print an extended help message, listing all options and any help text provided with them, to ‘file’ (default stdout).

print_manpage(file=None)

outputs a manpage for the program using the help information

print_usage(file : file = stdout)

Print the usage message for the current program (self.usage) to ‘file’ (default stdout). Any occurrence of the string “%prog” in self.usage is replaced with the name of the current program (basename of sys.argv[0]). Does nothing if self.usage is empty or not defined.

print_version(file : file = stdout)

Print the version message for this program (self.version) to ‘file’ (default stdout). As with print_usage(), any occurrence of “%prog” in self.version is replaced by the current program’s name. Does nothing if self.version is empty or undefined.

processfile(fileprocessor, options, fullinputpath, fulloutputpath, fulltemplatepath)

Process an individual file.

recurseinputfilelist(options)

Use a list of files, and find a common base directory for them.

recurseinputfiles(options)

Recurse through directories and return files to be processed.

recursiveprocess(options)

Recurse through directories and process files.

reportprogress(filename, success)

Shows that we are progressing...

run()

Parses the arguments, and runs recursiveprocess with the resulting options...

set_usage(usage=None)

sets the usage string - if usage not given, uses getusagestring for each option

seterrorleveloptions()

Sets the errorlevel options.

setformats(formats, usetemplates)

Sets the format options using the given format dictionary.

Parameters:formats (Dictionary) –

The dictionary keys should be:

  • Single strings (or 1-tuples) containing an input format (if not usetemplates)
  • Tuples containing an input format and template format (if usetemplates)
  • Formats can be None to indicate what to do with standard input

The dictionary values should be tuples of outputformat (string) and processor method.

setmanpageoption()

creates a manpage option that allows the optionparser to generate a manpage

setprogressoptions()

Sets the progress options.

splitext(pathname)

Splits pathname into name and ext, and removes the extsep.

Parameters:pathname (string) – A file path
Returns:root, ext
Return type:tuple
splitinputext(inputpath)

Splits an inputpath into name and extension.

splittemplateext(templatepath)

Splits a templatepath into name and extension.

templateexists(options, templatepath)

Returns whether the given template exists...

warning(msg, options=None, exc_info=None)

Print a warning message incorporating ‘msg’ to stderr and exit.

ourdom

module that provides modified DOM functionality for our needs

Note that users of ourdom should ensure that no code might still use classes directly from minidom, like minidom.Element, minidom.Document or methods such as minidom.parseString, since the functionality provided here will not be in those objects.

translate.misc.ourdom.getElementsByTagName_helper(parent, name, dummy=None)

A reimplementation of getElementsByTagName as an iterator.

Note that this is not compatible with getElementsByTagName that returns a list, therefore, the class below exposes this through yieldElementsByTagName

translate.misc.ourdom.getnodetext(node)

returns the node’s text by iterating through the child nodes

translate.misc.ourdom.parse(file, parser=None, bufsize=None)

Parse a file into a DOM by filename or file object.

translate.misc.ourdom.parseString(string, parser=None)

Parse a file into a DOM from a string.

translate.misc.ourdom.searchElementsByTagName_helper(parent, name, onlysearch)

limits the search to within tags occuring in onlysearch

translate.misc.ourdom.writexml_helper(self, writer, indent='', addindent='', newl='')

A replacement for writexml that formats it like typical XML files. Nodes are intendented but text nodes, where whitespace can be significant, are not indented.

profiling

translate.misc.profiling.profile_func(filename=None, mode='w+')

Function/method decorator that will cause only the decorated callable to be profiled (with a KCacheGrind profiler) and saved to the specified file.

Parameters:
  • filename (str) – The filename to write the profile to. If not specified the decorated function’s name is used, followed by _func.profile.
  • mode (str) – The mode in which to open :param:`filename`. Default is w+.

progressbar

Progress bar utilities for reporting feedback on the progress of an application.

class translate.misc.progressbar.DotsProgressBar

An ultra-simple progress indicator that just writes a dot for each action

show(verbosemessage)

show a dot for progress :-)

class translate.misc.progressbar.HashProgressBar(*args, **kwargs)

A ProgressBar which knows how to go back to the beginning of the line.

class translate.misc.progressbar.MessageProgressBar(*args, **kwargs)

A ProgressBar that just writes out the messages without any progress display

class translate.misc.progressbar.NoProgressBar

An invisible indicator that does nothing.

show(verbosemessage)

show nothing for progress :-)

class translate.misc.progressbar.ProgressBar(minValue=0, maxValue=100, totalWidth=50)

A plain progress bar that doesn’t know very much about output.

show(verbosemessage)

displays the progress bar

quote

String processing utilities for extracting strings with various kinds of delimiters

translate.misc.quote.escapecontrols(source)

escape control characters in the given string

translate.misc.quote.extract(source, startdelim, enddelim, escape=None, startinstring=False, allowreentry=True)

Extracts a doublequote-delimited string from a string, allowing for backslash-escaping returns tuple of (quoted string with quotes, still in string at end).

translate.misc.quote.extractwithoutquotes(source, startdelim, enddelim, escape=None, startinstring=False, includeescapes=True, allowreentry=True)

Extracts a doublequote-delimited string from a string, allowing for backslash-escaping includeescapes can also be a function that takes the whole escaped string and returns the replaced version.

translate.misc.quote.find_all(searchin, substr)

Returns a list of locations where substr occurs in searchin locations are not allowed to overlap

translate.misc.quote.htmlentitydecode(source)

decodes source using HTML entities e.g. &copy; -> ©

translate.misc.quote.htmlentityencode(source)

encodes source using HTML entities e.g. © -> &copy;

translate.misc.quote.javapropertiesencode(source)

Encodes source in the escaped-unicode encoding used by Java .properties files

translate.misc.quote.mozillapropertiesencode(source)

Encodes source in the escaped-unicode encoding used by Mozilla .properties files.

translate.misc.quote.propertiesdecode(source)

Decodes source from the escaped-unicode encoding used by .properties files.

Java uses Latin1 by default, and Mozilla uses UTF-8 by default.

Since the .decode(“unicode-escape”) routine decodes everything, and we don’t want to we reimplemented the algorithm from Python Objects/unicode.c in Python and modify it to retain escaped control characters.

selector

selector - WSGI delegation based on URL path and method.

(See the docstring of selector.Selector.)

Copyright (C) 2006 Luke Arno - http://lukearno.com/

This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

Luke Arno can be found at http://lukearno.com/

class translate.misc.selector.ByMethod

Base class for dispatching to method named by REQUEST_METHOD.

class translate.misc.selector.EnvironDispatcher(rules)

Dispatch based on list of rules.

class translate.misc.selector.MiddlewareComposer(app, rules)

Compose middleware based on list of rules.

class translate.misc.selector.Naked

Naked object style dispatch base class.

class translate.misc.selector.Selector(mappings=None, prefix='', parser=None, wrap=None, mapfile=None, consume_path=True)

WSGI middleware for URL paths and HTTP method based delegation.

See http://lukearno.com/projects/selector/

Mappings are given are an iterable that returns tuples like this:

(path_expression, http_methods_dict, optional_prefix)
add(path, method_dict=None, prefix=None, **http_methods)

Add a mapping.

HTTP methods can be specified in a dict or using kwargs, but kwargs will override if both are given.

Prefix will override self.prefix for this mapping.

select(path, method)

Figure out which app to delegate to or send 404 or 405.

slurp(mappings, prefix=None, parser=None, wrap=None)

Slurp in a whole list (or iterable) of mappings.

Prefix and parser args will override self.parser and self.args for the given mappings.

slurp_file(the_file, prefix=None, parser=None, wrap=None)

Read mappings from a simple text file.

Format looks like this:

{{{

# Comments if first non-whitespace char on line is '#'
# Blank lines are ignored

/foo/{id}[/]
    GET somemodule:some_wsgi_app
    POST pak.subpak.mod:other_wsgi_app

@prefix /myapp
/path[/]
    GET module:app
    POST package.module:get_app('foo')
    PUT package.module:FooApp('hello', resolve('module.setting'))

@parser :lambda x: x
@prefix 
^/spam/eggs[/]$
    GET mod:regex_mapped_app

}}}

@prefix and @parser directives take effect until the end of the file or until changed.

static status404(environ, start_response)

Respond with a 404.

static status405(environ, start_response)

Respond with a 405 and appropriate Allow header.

class translate.misc.selector.SimpleParser(patterns=None)

Callable to turn path expressions into regexes with named groups.

For instance "/hello/{name}" becomes r"^\/hello\/(?P<name>[^\^.]+)$"

For /hello/{name:pattern} you get whatever is in self.patterns['pattern'] instead of "[^\^.]+"

Optional portions of path expression can be expressed [like this]

/hello/{name}[/] (can have trailing slash or not)

Example:

/blog/archive/{year:digits}/{month:digits}[/[{article}[/]]]

This would catch any of these:

/blog/archive/2005/09
/blog/archive/2005/09/
/blog/archive/2005/09/1
/blog/archive/2005/09/1/

(I am not suggesting that this example is a best practice. I would probably have a separate mapping for listing the month and retrieving an individual entry. It depends, though.)

lastly(regex)

Process the result of __call__ right before it returns.

Adds the ^ and the $ to the beginning and the end, respectively.

lookup(name)

Return the replacement for the name found.

openended(regex)

Process the result of __call__ right before it returns.

Adds the ^ to the beginning but no $ to the end. Called as a special alternative to lastly.

outermost_optionals_split(text)

Split out optional portions by outermost matching delims.

parse(text)

Turn a path expression into regex.

translate.misc.selector.expose(obj)

Set obj._exposed = True and return obj.

translate.misc.selector.method_not_allowed(environ, start_response)

Respond with a 405 and appropriate Allow header.

translate.misc.selector.not_found(environ, start_response)

Respond with a 404.

translate.misc.selector.opliant(meth)

Decorate a bound wsgi callable taking args from wsgiorg.routing_args

class App(object):
    @opliant
    def __call__(self, environ, start_response, arg1, arg2, foo='bar'):
        ...
translate.misc.selector.pliant(func)

Decorate an unbound wsgi callable taking args from wsgiorg.routing_args

@pliant
def app(environ, start_response, arg1, arg2, foo='bar'):
    ...

sparse

simple parser / string tokenizer rather than returning a list of token types etc, we simple return a list of tokens. Each tokenizing function takes a string as input and returns a list of tokens.

exception translate.misc.sparse.ParserError(parser, message, tokennum)

Intelligent parser error

class translate.misc.sparse.SimpleParser(defaulttokenlist=None, whitespacechars=' trn', includewhitespacetokens=0)

this is a simple parser

applytokenizer(inputlist, tokenizer)

apply a tokenizer to a set of text, flattening the result

applytokenizers(inputlist, tokenizers)

apply a set of tokenizers to a set of text, flattening each time

findtokenpos(tokennum)

finds the position of the given token in the text

getlinepos(tokenpos)

finds the line and character position of the given character

isstringtoken(text)

checks whether a token is a string token

keeptogether(text)

checks whether a token should be kept together

raiseerror(message, tokennum)

raises a ParserError

removewhitespace(text)

this removes whitespace but lets it separate things out into separate tokens

separatetokens(text, tokenlist=None)

this separates out tokens in tokenlist from whitespace etc

stringtokenize(text)

makes strings in text into tokens...

tokenize(source, tokenizers=None)

tokenize the text string with the standard tokenizers

translate.misc.sparse.stringeval(text)

takes away repeated quotes (escapes) and returns the string represented by the text

translate.misc.sparse.stringquote(text)

escapes quotes as neccessary and returns a string representing the text

stdiotell

A wrapper for sys.stdout etc that provides tell() for current position

textwrap

Text wrapping and filling.

class translate.misc.textwrap.TextWrapper(width=70, initial_indent='', subsequent_indent='', expand_tabs=True, drop_whitespace=True, replace_whitespace=True, fix_sentence_endings=False, break_long_words=True)

Object for wrapping/filling text. The public interface consists of the wrap() and fill() methods; the other methods are just there for subclasses to override in order to tweak the default behaviour. If you want to completely replace the main wrapping algorithm, you’ll probably have to override _wrap_chunks().

Several instance attributes control various aspects of wrapping:
width (default: 70)
the maximum width of wrapped lines (unless break_long_words is false)
initial_indent (default: “”)
string that will be prepended to the first line of wrapped output. Counts towards the line’s width.
subsequent_indent (default: “”)
string that will be prepended to all lines save the first of wrapped output; also counts towards each line’s width.
expand_tabs (default: true)
Expand tabs in input text to spaces before further processing. Each tab will become 1 .. 8 spaces, depending on its position in its line. If false, each tab is treated as a single character.
drop_whitespace (default: true)
Drop leading and trailing whitespace from lines.
replace_whitespace (default: true)
Replace all whitespace characters in the input text by spaces after tab expansion. Note that if expand_tabs is false and replace_whitespace is true, every tab will be converted to a single space!
fix_sentence_endings (default: false)
Ensure that sentence-ending punctuation is always followed by two spaces. Off by default because the algorithm is (unavoidably) imperfect.
break_long_words (default: true)
Break words longer than ‘width’. If false, those words will not be broken, and some lines might be longer than ‘width’.
fill(text : string) → string

Reformat the single paragraph in ‘text’ to fit in lines of no more than ‘self.width’ columns, and return a new string containing the entire wrapped paragraph.

wrap(text : string) → [string]

Reformat the single paragraph in ‘text’ so it fits in lines of no more than ‘self.width’ columns, and return a list of wrapped lines. Tabs in ‘text’ are expanded with string.expandtabs(), and all other whitespace characters (including newline) are converted to space.

translate.misc.textwrap.wrap(text, width=70, **kwargs)

Wrap a single paragraph of text, returning a list of wrapped lines.

Reformat the single paragraph in ‘text’ so it fits in lines of no more than ‘width’ columns, and return a list of wrapped lines. By default, tabs in ‘text’ are expanded with string.expandtabs(), and all other whitespace characters (including newline) are converted to space. See TextWrapper class for available keyword args to customize wrapping behaviour.

translate.misc.textwrap.fill(text, width=70, **kwargs)

Fill a single paragraph of text, returning a new string.

Reformat the single paragraph in ‘text’ to fit in lines of no more than ‘width’ columns, and return a new string containing the entire wrapped paragraph. As with wrap(), tabs are expanded and other whitespace characters converted to space. See TextWrapper class for available keyword args to customize wrapping behaviour.

typecheck

doctest_support

This module allows doctest to find typechecked functions.

Currently, doctest verifies functions to make sure that their globals() dict is the __dict__ of their module. In the case of decorated functions, the globals() dict is not the right one.

To enable support for doctest do:

import typecheck.doctest_support

This import must occur before any calls to doctest methods.

mixins

sets

typeclasses

wsgi

Wrapper to launch the bundled CherryPy server.

translate.misc.wsgi.launch_server(host, port, app, **kwargs)

Use CherryPy’s WSGI server, a multithreaded scallable server.

wStringIO

A wrapper for cStringIO that provides more of the functions of StringIO at the speed of cStringIO

class translate.misc.wStringIO.CatchStringOutput(onclose)

catches the output before it is closed and sends it to an onclose method

close()

wrap the underlying close method, to pass the value to onclose before it goes

slam()

use this method to force the closing of the stream if it isn’t closed yet

xml_helpers

Helper functions for working with XML.

translate.misc.xml_helpers.getText(node, xml_space='preserve')

Extracts the plain text content out of the given node.

This method checks the xml:space attribute of the given node, and takes an optional default to use in case nothing is specified in this node.

translate.misc.xml_helpers.getXMLlang(node)

Gets the xml:lang attribute on node

translate.misc.xml_helpers.getXMLspace(node, default=None)

Gets the xml:space attribute on node

translate.misc.xml_helpers.namespaced(namespace, name)

Returns name in Clark notation within the given namespace.

For example namespaced(“source”) in an XLIFF document might return::
{urn:oasis:names:tc:xliff:document:1.1}source

This is needed throughout lxml.

translate.misc.xml_helpers.normalize_space(text)

Normalize the given text for implementation of xml:space="default".

translate.misc.xml_helpers.normalize_xml_space(node, xml_space, remove_start=False)

normalize spaces following the nodes xml:space, or alternatively the given xml_space parameter.

translate.misc.xml_helpers.setXMLlang(node, lang)

Sets the xml:lang attribute on node

translate.misc.xml_helpers.setXMLspace(node, value)

Sets the xml:space attribute on node

translate.misc.xml_helpers.string_xpath = string()

Return a non-normalized string in the node subtree

translate.misc.xml_helpers.string_xpath_normalized = normalize-space()

Return a (space) normalized string in the node subtree

translate.misc.xml_helpers.xml_preserve_ancestors = ancestor-or-self::*[attribute::xml:space='preserve']

All ancestors with xml:space=’preserve’

translate.misc.xml_helpers.xml_space_ancestors = ancestor-or-self::*/attribute::xml:space

All xml:space attributes in the ancestors

xmlwrapper

zipfileext

Extensions to zipfile standard module that will hopefully get included in future.

class translate.misc.zipfileext.ZipFileExt(file, mode='r', compression=0, allowZip64=False)

a ZipFile that can handle replacing objects

close()

Close the file, and for mode “w” and “a” write the ending records.

comment

The comment text associated with the ZIP file.

delete(name)

Delete the file from the archive. If it appears multiple times only the first instance will be deleted.

extract(member, path=None, pwd=None)

Extract a member from the archive to the current working directory, using its full name. Its file information is extracted as accurately as possible. `member’ may be a filename or a ZipInfo object. You can specify a different directory using `path’.

extractall(path=None, members=None, pwd=None)

Extract all members from the archive to the current working directory. `path’ specifies a different directory to extract to. `members’ is optional and must be a subset of the list returned by namelist().

getinfo(name)

Return the instance of ZipInfo given ‘name’.

infolist()

Return a list of class ZipInfo instances for files in the archive.

namelist()

Return a list of file names in the archive.

open(name, mode='r', pwd=None)

Return file-like object for ‘name’.

printdir()

Print a table of contents for the zip file.

read(name, pwd=None)

Return file bytes (as a string) for name.

setpassword(pwd)

Set default password for encrypted files.

testzip()

Read all the files and check the CRC.

write(filename, arcname=None, compress_type=None)

Put the bytes from filename into the archive under the name arcname.

writeendrec()

Write the ending records (without neccessarily closing the file)

writestr(zinfo_or_arcname, bytes, compress_type=None)

Write a file into the archive. The contents is the string ‘bytes’. ‘zinfo_or_arcname’ is either a ZipInfo instance or the name of the file in the archive.