agutil 4.1.0

• Added 3 new classes to serve as argparse argument type helpers when dealing with files and directories:
  • agutil.FileType accepts filenames while checking for valid file extensions and optional compression extensions
  • agutil.DirType accepts directory names
  • agutil.FOFNType accepts files containing file names (FOFN) while checking that the listed files have valid extensions and optional compression extensions
• Callback objects returned by agutil.parallel.ThreadWorker and agutil.parallel.ProcessWorker now have the following new attributes
  • func : The function executed
  • args : The positional arguments given
  • kwargs : The keyword arguments given
  • poll() : A function which returns True when the background function has completed execution and is ready to return a result

agutil 4.0.2

• Fixed the progress bar not displaying properly when encrypting/decrypting multiple files at the same time using agutil-secure
• Fixed agutil.parallel.ProcessWorker not raising exceptions until all other jobs had finished. Exceptions will now be raised immediately and shutdown the ProcessWorker

agutil 4.0.1

• Set the file argument to the agutil.status_bar constructor to default to None. During initialization, if file is None, it becomes sys.stdout. This allows
  status_bar to be affected by changes to sys.stdout

agutil 4.0.0

Feature Removal:

• The maf2bed utility and its associated code has been removed

Deprecation Notice:

• agutil.io.QueuedSocket is now deprecated in favor of agutil.io.MPlexSocket

Breaking Changes:

• agutil.Logger has been rewritten to integrate with the builtin logging module
• The constructor for agutil.io.QueuedSocket no longer takes an agutil.io.Socket as argument, but instead takes an address and port, like agutil.io.Socket
• Removed the -f\--force flag from agutil-secure
• Changed the arguments for agutil.security.encryptFile, agutil.security.encryptFileObj, agutil.security.decryptFile, and agutil.security.decryptFileObj to support new ciphers
• The constructor for agutil.io.SecureSocket no longer takes an agutil.io.Socket as an argument, but instead takes an address and port, like agutil.io.Socket
• agutil.security.SecureConnection has been overhauled to be more efficient, and mostly threadless. It is now incompatible with previous versions of the class
  • The retries argument of the send() function has been removed
  • Removed the timeout argument from close() and shutdown()

New Features:

• Added agutil.ActiveTimeout class to manage a timeout over a set of operations. Specifically, to enforce that several blocking operations all complete within the given timeout. The enforcement is not automatic, but relies on checking the timeout when possible
• agutil.split_iterable is now also available as agutil.clump which was chosen as it more clearly describes the function. agutil.split_iterable may be removed in a future release, but this is not yet planned
• Added a function agutil.splice() which takes a iterable of at least 2 dimensions (M rows by N columns)
  and returns an iterable for each column (N iterables of length M)
• Added agutil.context_lock() method, which enforces a timeout when acquiring a native lock
• Added the following exceptions:
  • agutil.LockTimeoutExceeded
  • agutil.TimeoutExceeded
  • agutil.security.CipherError
  • agutil.security.HeaderError
  • agutil.security.HeaderLengthError
  • agutil.security.InvalidHeaderError
  • agutil.security.EncryptionError
  • agutil.security.Decryptionerror
• Added a passthrough() method to agutil.status_bar, which takes one argument and returns it unchanged while incrementing the bar by 1
• Added agutil.io.MPlexSocket class. This class provides the same interface as agutil.io.QueuedSocket but does so without the use of background threads. It is meant to serve as a drop-in replacement, but due to the significance of the change, it was written as a new class. agutil.security.SecureSocket now derives from this class
• Added agutil.security.encryptFileObj() and agutil.security.decryptFileObj()
  methods. These methods take the same arguments as agutil.security.encryptFile
  and agutil.security.decryptFile methods except that they take file-like objects
  instead of filenames
• Added a confirm() method to agutil.security.SecureSocket which awaits confirmation from the remote socket that a task has been completed
• Added the agutil.security.EncryptionCipher class, which is a configurable cipher that provides the AES encryption backend for agutil-secure and the agutil.security module
• Added the agutil.security.DecryptionCipher class, which is a configurable cipher that provides the AES decryption backend for agutil-secure and the agutil.security module
• Added the agutil.security.CipherHeader class, which stores the configuration of an EncryptionCipher or DecryptionCipher and can be used to produce the 16-byte cipher header representing the configuration
• Added agutil.security.configure_cipher(), a utility method for generating an agutil.security.CipherHeader based on settings provided as keyword arguments
• Added the agutil.security.Bitmask class to allow reading and manipulating a single-byte bitmask using python's [] indexing API. This class uses an agutil.search_range to perform underlying bit manipulations and queries
• Added adispatch() method to agutil.parallel.IterDispatcher as the preferred method for operating the dispatcher, but operates identically to IterDispatcher.run()

Other Changes:

• agutil.byteSize now supports yottabytes
• Added a file argument to the agutil.status_bar() constructor
• The SocketServer.accept() method of agutil.io.SocketServer now takes a socket_type argument to determine which type of Socket in the agutil socket class tree will be returned. The method also takes a variable number of keyword arguments which will be passed to the returned socket's constructor. The method defaults to returning agutil.io.Socket instances
• Outgoing messages of an agutil.io.QueuedSocket instance are no longer sent in a FIFO order. The socket will continuously rotate through all channels waiting to send messages, sending one message from each (in FIFO)
• Updated cipher used by agutil-secure
  • Capable of decrypting data encrypted by agutil >= 1.2.0 by default
  • Capable of encrypting/decrypting data compatible with agutil <= 1.1.2 by using the -l\--legacy flag
• agutil.io.SecureSocket now derives from agutil.io.MPlexSocket instead of agutil.io.QueuedSocket
• Added sign argument to the sendRSA() method of agutil.io.SecureSocket
  • SecureScoket.sendRSA() and SecureSocket.recvRSA() now handle signature validation internally
• agutil.security.SecureSocket now uses agutil.security.EncryptionCipher and agutil.security.EncryptionCipher as backends for sendAES() and recvAES()
• The WORKERTYPE constants, agutil.parallel.WORKERTYPE_THREAD and agutil.parallel.WORKERTYPE_PROCESS, have been updated to reference the actual worker classes, agutil.parallel.ThreadWorker and agutil.parallel.ProcessWorker, instead of being enumerations for the types. The consequence is that the parallelization decorators and dispatchers can now be given any class which follows the worker interface
• The workertype argument to the constructors of agutil.parallel.IterDispatcher and agutil.parallel.DemandDispatcher may now be any object which follows the worker interface. It still defaults to a ThreadWorker

agutil 3.1.2

• Fixed setup.py to allow installation using pip >=10 and python <3.6

agutil 3.1.1

• Switched from PyCrypto to PyCryptodomeX:
  • The former library is no longer maintained
  • PyCryptodomeX is not vulnerable to CVE-2018-6594
  • PyCryptodomeX was chosen over PyCryptodome to force compliance and avoid conflicts

agutil 3.1.0

Deprecation Warning:

• The maf2bed utility, and its associated code, are now deprecated and will be
  removed in a future release

agutil 3.0.1

Note: This version was released on May 3rd, 2018

• Fixed setup.py to allow source builds on pip 10

agutil 3.0.0

Note: This version was released on Febuary 9th, 2018

Breaking changes:

• agutil.parallel.parallelize and agutil.parallel.parallelize2 must now be called when used as decorators

---

Other changes:

• Added new agutil.first() function to return the first element of an iterable matching a given predicate
• The padding scheme used in the agutil.security module has been changed to be more cryptographically secure while maintaining compatibility with the old padding scheme
• The value argument to agutil.status_bar.update() is now optional. If omitted, the status bar will be incremented by one
• Added a new agutil.status_bar.iter() class method. The goal of this function is to serve as a wrapper for iterables to add a status bar to any loop
• agutil.parallel.parallelize() and agutil.parallel.parallelize2() now take a workertype argument to set if parallelization will be thread or process based.
• agutil.parallel.Dispatcher has been refactored and renamed to agutil.parallel.IterDispatcher, but mostly follows the same syntax
• Added agutil.parallel.DemandDispatcher to be the dispatching backend for agutil.parallel.parallelize2 instead of relying on heavy logic within the function
• Added agutil.parallel.ThreadWorker to manage worker threads for thread-based parallelization
• Added agutil.parallel.ProcessWorker to manage worker processes for process-based parallelization

agutil 2.1.2

• Fixed agutil.byteSize returning an incorrect result if the output ended with '0'