ship.utils package

Submodules

ship.utils.atool module

Summary:

Contains the ATool class an interface that all tools (i.e. file loaders, etc) should inherit.

This was a integral part of these tools when they were written in Java as they allowed the user interface to communicate with the tools through an observer. In Python it’s not needed because of the signal slot mechanism making everything a lot easier.

The interface is kept partly because it may still be useful and partly because I’m sure there will be a need to have all similar tools grouped at some point.

Author:
Duncan Runnacles
Created:
01 Apr 2016
Copyright:
Duncan Runnacles 2016

TODO:

Updates:

class ATool[source]

Bases: object

Abstract base class for all tools.

Any class that functions as a tool for processing the files etc should implement this.

Note

This is not currently used in any part of the software at the moment. It was needed in the original Java implementation in order to allow tools to talk to the Gui through an aobserver interface.

It is here in case it’s needed but it is probably unnecessary because of the signal-slot mechanism of the Qt library which makes this kind of thing very simple.

Re-visit whether an Observer pattern is required once we have some threading setup. Although Qt does that all so well I very much doubt that this interface is required.

init(observer=None)[source]

Constructor.

Takes in an instance of an observer to register it.

Parameters:observer (object) – the class that want to be notified.
notifyObserversProgress(progress)[source]

Notify all the current observers of a progress update

notifyObserversProgressName(name)[source]

Notify all Observers of the name of the current progress.

This name may also be a description. It is used to display on the user interface to tell users what is happening when this tools is called.

Parameters:name (str) – the name of the progress.
registerObserver(observer)[source]

Add an observer to the collection of observers in the list.

Parameters:observer (object) – object to add to the observer list
unregisterObserver(o)[source]

Remove an observer from the list of observers.

Parameters:o (object) – an object that no longer wants to be notified.
updateObserversTotal(progress)[source]

Notify all of the current observers of what we expect the total progress to be.

Parameters:progress (int) – The total progress expected.
logger = <logging.Logger object at 0x04375A10>

logging references with a __name__ set to this module.

ship.utils.filetools module

Summary:

Contains file access functions and classes.

This module contains convenience functions and classes for reading and writing files and launching different types of File dialogue.

Some of the functionality needed in this class is provided by the PyQt framework. When this module is loaded it tries to import the PyQt module. If it fails it sets a HAS_QT flag to False.

Before any attempts to use the PyQt functions are made the HAS_QT flag is checked. If it’s false the function will react.

Author:
Duncan Runnacles
Created:
01 Apr 2016
Copyright:
Duncan Runnacles 2016
TODO: This module, like a lot of other probably, needs reviewing for how

‘Pythonic’ t is. There are a lot of places where generators, comprehensions, maps, etc should be used to speed things up and make them a bit clearer.

More importantly there are a lot of places using ‘==’ compare that should be using ‘in’ etc. This could cause bugs and must be fixed soon.

Updates:

HAS_QT = False

If Qt is not installed on the machine running the library we can’t use any of the GUI related code, such as file dialogs. This flag informs the code in the module about the load status.

class PathHolder(path, root=None)[source]

Bases: object

Class for storing file paths.

This holds a range of useful path variables, such as: # file name # directory # full path # extension #etc

Contains functions useful for extracting and updating parts of a file path.

ABSOLUTE = 0
DIRECTORY = 2
EXTENSION = 4
NAME = 3
NAME_AND_EXTENSION = 5
RELATIVE = 1
absolutePath(filename=None, relative_roots=[], normalize=True)[source]

Get the absolute path of the file path stored in this object.

If there is no root variables set it will return False because there is no way of knowing what the absolute path will be.

Parameters:filename – Optional - if a different file name to the one currently stored in this object is required it can be specified with this variable.
Returns:str - absolute path of this object.
directory()[source]

Get the directory of the file path stored in this object.

This makes sure that the correct path, taking into consideration any relative path links to the main root, is returned.

If there is no root variable set it will return False because there is no way of knowing what the absolute path should be.

Returns:
str - absolute path of this object without the file name, or
False if no root variables have been set.
filenameAndExtension()[source]

Return the file name with the file extension appended.

Returns:
str - file name with the extension appended or a blank string if
the file name does not exist.
finalFolder()[source]

Get the last folder in the path.

If the relative_root variable is set it will be taken from that. Otherwise if it is not and the root variable is set it will be taken from that. If nether are set it will return False.

Returns:
tuple - the first part of the directory path and the final
folder in the directory path for this file, or False if the variable has not been set.
pathExists(ext=None)[source]

Test whether the path exists..

Note

If not self.root variable is set for this object then it is impossible to know if the path exists or not, so it will always return False.

Returns:
True if the path exists or False if it cannot be found or there
has not been a root variable set for this object.
relativePath(with_extension=True, filename=None)[source]

Returns the full relative root with filename for this object.

If a relative root was given to the constructor or set later this will return it, otherwise it will return False.

Parameters:
  • with_extension=True (bool) – append the extension to the end of the path if True.
  • filename=None (str) – if a different file name to the one currently stored in this object is required it can be specified with this variable.
Returns:

str - relative path of this object or False if there is no

relative path set.
setAbsolutePath(absolute_path, keep_relative_root=False)[source]

Sets the absolute path of this object.

Takes an absolute path and set the file variables in this object with it.

Note

This function WILL NOT check that the path exists. If the path used to call this function must exist it is the responsibility of the caller to check this.

keep_relative_root variable information:

If a relative root is not set to None it will be included in any path that is returned by this object. i.e. when an absolute path is returned by this object it will return:

path + relativepath + filename + extension

Parameters:
  • absolute_path (str) – the new absolute path to use to set the file variables in this object.
  • keep_relative_root=False (Bool) – if a relative root exists for this object it will be set to None unless this variable is set to True.
setFilename(filename, has_extension=False, keep_extension=False)[source]

Updates the filename variables with the given file name.

The caller can provide a filename with or without an extension and choose whether to keep the extension on the file or not.

has_extension will only be checked if keep_extension == False.

Parameters:
  • filename (str) – the new filename to set.
  • has_extension=False (Bool) – whether the filename has a file extension or not. This should be set to True if an extension exists. Otherwise the filename will be corrupted.
  • keep_extension – if the caller want to keep the extension on the given filename this should be True. Otherwise the file extension on the new filename will be discarded.
setFinalFolder(folder_name)[source]

Changes the final folder in the directory path to the given name

Parameters:folder_name (str) – the new name for the final folder in the path.
directory(in_path)[source]

Get the directory component of the given path.

Checks the given path to see if it has a file or not.

If there is no file component in the path it will return the path as-is.

If there is no directory component in the file it will return a blank string.

Parameters:in_path (str) – the path to extract the directory from.
Returns:Str - directory component of the given file path.

Todo

Is there any way to tell if it’s only a directory without being able to check that it exists and it doesn’t end in a slash?

directoryDialog(path='')[source]

Launch a file dialog and return the user selected directory.

If PyQt libraries are not available (HAS_QT == False) this function will raise an ImportError.

Parameters:path (str) – Optional - directory to open the dialog in.
Returns:Str containing path is successful or False if not.
Raises:ImportError – if PyQt libraries are not available.
directoryFileNames(path)[source]

Get a list of filenames in a directory.

Parameters:

path (str) –

Returns:

List - filenames in directory.
Raises:
  • IOError – If there’s a problem with the read/write or the directory doesn’t exist
  • TypeError – if string not given for file_path.
finalFolder(path)[source]

Get the last folder in the path.

Parameters:path (str) – the path to extract the final folder from.
Returns:str containing the name of the final folder in the path.
getFile(file_path)[source]

Text file reader.

Reads a text file, appending each new line to a list.

Parameters:

filePath (str) – File path for text file to load.
Returns:

List - contents of text file split by new lines.
Raises:
  • IOError – if problem in reading file.
  • TypeError – if string not given for file_path
getFileName(in_path, with_extension=False)[source]

Return the file name with the file extension appended.

Parameters:
  • in_path (str) – the file path to extract the filename from.
  • with_extension=False (Bool) – flag denoting whether to return the filename with or without the extension.
Returns:

Str - file name, with or without the extension appended or a

blank string if there is no file name in the path.
getOpenFileDialog(path='', types='All (*.*)', multi_file=True)[source]

Launch an open file dialog and return the user selected path.

The file types should follow the format:
‘DAT (.DAT);;TXT (.txt);;IEF (*.IEF)’

If PyQt libraries are not available (HAS_QT == False) this function will raise an ImportError.

Parameters:
  • path (str) – Optional - directory to open the dialog in.
  • types (str) – Optional - file types to restrict to.
  • multi_file=True (bool) – If False user can only select a single file.
Returns:

Str containing path is successful or False if not.
Raises:

ImportError – if PyQt libraries are not available.
getSaveFileDialog(path='', types='All (*.*)')[source]

Launch a save file dialog and return the user selected path.

The file types should follow the format:
“DAT (.DAT);;TXT (.txt);;IEF (*.IEF)”

If PyQt libraries are not available (HAS_QT == False) this function will raise an ImportError.

Parameters:
  • path (str) – Optional - directory to open the dialog in.
  • types (str) – Optional - file types to restrict to.
Returns:

Str containing path is successful or False if not.
Raises:

ImportError – if PyQt libraries are not available.
logger = <logging.Logger object>

logging references with a __name__ set to this module.

pathExists(path)[source]

Test whether a path exists.

Parameters:path (str) – the path to test.
Returns:True if the path exists or False if it doesn’t.
setFinalFolder(path, folder_name)[source]

Changes the final folder in the directory path to the given name.

Parameters:
  • path (str) – the path to update.
  • folder_name (str) – the new name for the final folder in the path.
Returns:

str containing the updated path.
writeFile(contents, file_path, add_newline=True)[source]

Text file writer

Writes a list to file, adding a new-line add the end of each list item.

Args:
contents (List) - lines to be written. filename (str) - Name of file to create. add_newline=True (Bool): adds a ‘
‘ to the end of each line written
if set to True.
Raises:
IOError: if problem in reading file. TypeError: if string not given for file_path

ship.utils.log module

Summary:
Central logging setup for the whole library. At the moment this is pretty simplistic. It just contains a simple formatter and allows for setting a global library debug level.
Author:
Duncan Runnacles
Created:
01 Apr 2016
Copyright:
Duncan Runnacles 2016

TODO:

Updates:

setup_custom_logger(name)[source]

ship.utils.qtclasses module

ship.utils.utilfunctions module

Summary:

Utility Functions that could be helpful in any part of the API.

All functions that are likely to be called across a number of classes and Functions in the API should be grouped here for convenience.

Author:
Duncan Runnacles
Created:
01 Apr 2016
Copyright:
Duncan Runnacles 2016
TODO: This module, like a lot of other probably, needs reviewing for how

‘Pythonic’ t is. There are a lot of places where generators, comprehensions, maps, etc should be used to speed things up and make them a bit clearer.

More importantly there are a lot of places using ‘==’ compare that should be using ‘in’ etc. This could cause bugs and must be fixed soon.

Updates:

class FileQueue[source]

Bases: object

Queueing class for storing data to go into the database

dequeue()[source]

Pop an item from the front of the queue.

enqueue(item)[source]

Add an item to the queue

isEmpty()[source]

Returns True if list is empty

size()[source]

Get the size of the queue

class LoadStack(max_size=-1)[source]

Bases: object

Stack class for loading logic.

add(item)[source]

Add an item to the stack.

Parameters:item – the item to add to the stack.
Raises:IndexError – if max_size has been set and adding another item would make the stack bigger than max size.
isEmpty()[source]

Return True if stack is empty.

peek()[source]

See what the next item on the stack is, but don’t remove it.

Returns:item from the top of the stack.
Raises:IndexError – if the stack is empty.
pop()[source]

Get an item From the stack.

Returns:item from the top of the stack.
Raises:IndexError – if the stack is empty.
size()[source]

Return the number of items in the stack.

arrayToString(self, str_array)[source]

Convert a list to a String

Creates one string by adding each part of the array to one string using ‘, ‘.join()

Parameters:str_array (List) – to convert into single string.
Returns:str - representaion of the array joined together.
Raises:ValueError – if not contents of list are instances of basestring.
checkFileType(file_path, ext)[source]

Checks a file to see that it has the right extension.

Parameters:
  • file_path (str) – The file path to check.
  • ext (List) – list containing the extension types to match the file against.
Returns:

True if the extension matches the ext variable given or False if not.
convertRunOptionsToSEDict(options)[source]

Converts tuflow command line options to scenario/event dict.

Tuflow uses command line option (e.g. -s1 blah -e1 blah) to set scenario values which can either be provided on the command line or through the FMP run form. The TuflowLoader can use these arguments but requires a slightly different setup.

This function converts the command line string into the scenarion and event dictionary expected by the TuflowLoader.

Parameters:options (str) – command line options.
Returns:{‘s1’: blah}, ‘event’: {‘e1’: blah}}
Return type:dict - {‘scenario’
Raises:AttributeError – if both -s and -s1 or -e and -e1 occurr in the options string. -x and -x1 are treated as the same variable by tuflow and one of the values would be ignored.
encodeStr(value)[source]
enum(*sequential, **named)[source]

Creates a new enum using the values handed to it.

Taken from Alec Thomas on StackOverflow: http://stackoverflow.com/questions/36932/how-can-i-represent-an-enum-in-python

Examples

Can be created and accessed using:

>>> Numbers = enum('ZERO', 'ONE', 'TWO')
>>> Numbers.ZERO
0
>>> Numbers.ONE
1

Or reverse the process o get the name from the value:

>>> Numbers.reverse_mapping['three']
'THREE'
fileExtensionWithoutPeriod(filepath, name_only=False)[source]

Extracts the extension without ‘.’ from filepath.

The extension will always be converted to lower case before returning.

Parameters:
  • filepath (str) – A full filepath if name_only=False. Otherwise a file name with extension if name_only=True.
  • name_only (bool) – True if filepath is only filename.extension.
findMax(val1, val2)[source]

Returns tuple containing min, max of two values

Parameters:
  • val1 – first integer or float.
  • val2 – second integer or float.
Returns:

  • lower value
  • higher value
  • False if not same or True if the same.
Return type:

tuple - containing
findSubstringInList(substr, the_list)[source]

Returns a list containing the indices that a substring was found at.

Uses a generator to quickly find all indices that str appears in.

Parameters:
  • substr (str) – the sub string to search for.
  • the_list (List) – a list containing the strings to search.
Returns:

  • a list with the indices that the substring was found in

    (this list can be empty if no matches were found).

  • an integer containing the number of elements it was found in.
Return type:

tuple - containing
findWholeWord(w)[source]

Find a whole word amoungst a string.

formatFloat(value, no_of_dps, ignore_empty_str=True)[source]

Format a float as a string to given number of decimal places.

Parameters:
  • value (float) – the value to format.
  • no_of_dps (int) – number of decimal places to format to.
  • ignore_empty_str (True) – return a stripped blank string if set to True.
Returns:

str - the formatted float.
Raises:

ValueError - if value param is not type float.
getSEResolvedFilename(filename, se_vals)[source]

Replace a tuflow placeholder filename with the scenario/event values.

Replaces all of the placholder values (e.g. ~s1~_~e1~) in a tuflow filename with the corresponding values provided in the run options string. If the run options flags are not found in the filename their values will be appended to the end of the string.

The setup of the returned filename is always the same:
  • First replace all placeholders with corresponding flag values.
  • s1 == s and e1 == e.
  • Append additional e values to end with ‘_’ before first and ‘+’ before others.
  • Append additional s values to end with ‘_’ before first and ‘+’ before others.
Parameters:
  • filename (str) – the filename to update.
  • se_vals (str) – the run options string containing the ‘s’ and ‘e’ flags and their corresponding values.
Returns:

str - the updated filename.
isList(value)[source]

Test a given value to see if it is a list or not.

Parameters:value – the variable to test for list type.
Returns:True if value is of type list; False otherwise.
isNumeric(s)[source]

Tests if string is a number or not.

Simply tries to convert it and catches the error if launched.

Parameters:s (str) – string to test number compatibility.
Returns:Bool - True if number. False if not.
isString(value)[source]

Tests a given value to see if it is an instance of basestring or not.

Note

This function should be used whenever testing this as it accounts for both Python 2.7+ and 3.2+ variations of string.

Parameters:value – the variable to test.
Returns:Bool - True if value is a unicode str (basestring type)
logger = <logging.Logger object at 0x0437DD30>

logging references with a __name__ set to this module.

Module contents