forked from troelsfr/BatchQ
-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathREADME
57 lines (46 loc) · 2.19 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
Installation
============
To install BatchQ type:
$ easy_install batchq
Documentation
=============
The documentation can be found at http://batchq.rtfd.org
RENAMING AND MOVING
===================
* (V) Shell.status -> shell.state
* (V) Shell.status: return shell.state.texts[shell.state]
* batchq.core.Shell -> batchq.core.QueueElement
* batchq.core.Collection -> batchq.core.JobList
* FileCommander -> batchq.core.batch.Connection, batchq.core.batch.Machine or something
* batchq.core.batch -> batchq.batch.__init__
* end_machine -> previous_machine(), deselect_machine()
TODO
====
* There might occur an error when large output are retrieved (len(buf) > 10000 )
* Add SHA/MD5 ID generation to the batch modules
* Combine objects of the same kind
* Add caching on the status call
* (V) Add file caching to submission modules
* Move batchq files to a central location
OPTIMISATIONS
=============
* Reduce the amount of calls of read_non_blocking_until
* Reduce the amount of calls to writeChar / put
* Many calls to base.py:274(fix_buffer)
- Remove len from fix buffer and you will reduce the calls to len by a factor 1/2 and save ~0.5 sec pr. run call.
- Last half of the function can be deleted if empty lines are always appended in the size of the line length
* Also many calls to patternregister.py:152(parse)
* Make Shell + Shell, Subshell + Subshell and LSF + LSF reduable to a script
* Checkout wether it is possible to introduce additional caching
BUGS
====
* Following input:
400075#-->if [ -f '.batchq.qa_main-16qubit_975_400paramh5-16qubit_975_400outh5.p then echo "SUCCESS"; fi
SUCCESS
is interpreted as
#-->cat .batchq.qa_main-16qubit_950_400paramh5-16qubit_950_400outh5.pid
400075#-->if [ -f '.batchq.qa_main-16qubit_975_400paramh5-16qubit_975_400outh5.p then echo "SUCCESS"; fi
SUCCESS
* In the LSF submission commmand, if no ".batchq.id.finished' exists, if [ $(cat blah ) -ne 0 ] fails. Add a check wether the file exists first. If not create .failed
* Commands with "find some*" are wrong and should be replaced with their regex equivalents
* When things are subshelled you may run into problems with output that arrives asyncronously. This means that consume output will fail.