Skip to content

FontysVenlo/codestripper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

98 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CodeStripper

New version of the CodeStripper that was previously used, which can be found at https://github.com/sebivenlo/codestripper.

The reason for the switch is to not be dependent on Ant as a build system and the possibility to easily add more tags.

Available tags

Command Tag(s) Description
Add cs:add:<text> Add the text (without the tag in front)
Ignore cs:ignore Ignore the entire file, only valid on the first line
Remove line cs:remove Remove the line
Remove range cs:remove:start/cs:remove:end Remove all text in between tags
Replace cs:replace:<replacement> Replace the text in front by the replacement, keeps whitespace
Uncomment cs:uncomment:start/cs:uncomment:end Uncomment all lines in between tags

Legacy

To support the old CodeStripper, the legacy tag Start Solution::replacewith::/End Solution::replacewith:: is still supported for now. This tag does both the Remove and Replace in one go.

Command Line Properties

CodeStripper can be used as a Python Module and as a command line tool. The command line tool has the following interface.

Flag Long form Description Default value Required
<positional> None files to include for code stripping (glob) None True
-e --exclude files to exclude for code stripping (glob) None False
-c --comment comment symbol(s) for the given language // False
-o --output the output directory to store the stripped files out False
-r --recursive do NOT use recursive globs for include/exclude True False
-v --verbosity increase output verbosity None False
-d --dry execute a dry run False False
-w --working-directory set the working directory for include/exclude pwd False

Examples

This section contains examples for all supported tags.

Add

Input:

public class Test {
    //cs:add:private final String test = "test";
}

Output:

public class Test {
    private final String test = "test";
}

Ignore

Input:

//cs:ignore
public class Test {
    private final String test = "test";
}

Output: No output, file is ignored

Remove line

Input:

public class Test {
    private final String test = "test";//cs:remove
}

Output:

public class Test {
}

Remove range

Input:

public class Test {
    //cs:remove:start
    private final String test = "test";
    private final int count = 0;
    //cs:remove:end
    private final boolean keep = true;
}

Output:

public class Test {
    private final boolean keep = true;
}

Replace

Input:

public class Test {
    private final boolean keep = false;//cs:replace://TODO: add fields
}

Output:

public class Test {
    //TODO: add fields
}

Uncomment

Input:

public class Test {
    //cs:uncomment:start
    //private final String example = "example";
    //private final boolean isTestCode = true;
    //cs:uncomment:end
}

Output:

public class Test {
    private final String example = "example";
    private final boolean isTestCode = true;
}

Adding a new tag

It is possible to add custom tags. There a two types of tags: SingleTag that works on one line only and RangeTag that works on a range of lines. Tags are defined as follows:

classDiagram
    Tag <|-- SingleTag
    SingleTag <|-- RangeOpenTag
    SingleTag <|-- RangeCloseTag
    Tag <|-- RangeTag
    
    class Tag{
        <<Abstract>>
        +offset: int
        +start: int
        +end: int
        +is_valid()*: bool
        +execute()*: Optional[str]
    }
    class SingleTag{
        +regex: str
        +param_start: int
        +param_end: int
        +regex_start: int
        +regex_end: int
        +leading_characters: str
        +parameter: str
        +whitespace: str
        +SingleTag(data: TagData)
    }
    class RangeOpenTag{
        +RangeOpen(parent: Type, data: TagData)
    }
    class RangeCloseTag{
        +RangeOpen(parent: Type, data: TagData)
    }
    class RangeTag{
        +inset: int
        +start: int
        +end: int
        +open_tag: RangeOpenTag
        +close_tag: RangeCloseTag
        +RangeTag(open_tag: RangeOpenTag, close_tag: RangeCloseTag)
        +add_tags(tags: Iterable[Tag])
    }
Loading

The idea is that every tag has the following methods:

  • is_valid: whether the tag is valid
  • execute: handle the text for this tag

RangeTag work in the following way:

  • RangeOpenTag: Specifies the open regex and handles the opening line. Defines the type of parent it belongs to (so that the tokenizer can match open and close tag)
  • RangeCloseTag: Specifies the close regex and handles the closing line. Defines the type of parent it belongs to (so that the tokenizer can match open and close tag)
  • RangeTag: Handles lines in between the open and close tag. Has access to both the open and close tag that were matched by the tokenizer.

Create a custom tag:

  1. Create a new file for your tag(s)
  2. Depending on if you create a SingleTag or RangeTag
  • SingleTag:
class TestTag(SingleTag):
    regex = r'<regex>' # Regex that should match the tag

    def __init__(self, data: TagData) -> None:
        super().__init__(data)

    def execute(self, content: str) -> Optional[str]:
        # Manipulate the line
        # None means the line is removed
    
    def is_valid(self) -> bool:
        # Return wether the tag is valid
  • RangeTag: Range needs a RangeOpenTag, RangeCloseTag and a RangeTag
class TestOpenTag(RangeOpenTag):
    regex = r'<regex>' # Regex that should match the open tag

    def __init__(self, data: TagData) -> None:
        super().__init__(TestRangeTag, data)# Type of RangeTag is belong to

    def execute(self, content: str) -> Optional[str]:
        # Manipulate the line
        # None means the line is removed
    
    def is_valid(self) -> bool:
        # Return wether the tag is valid

class TestCloseTag(RangeCloseTag):
    # Same as RangeOpenTag

class TestRangeTag(RangeTag):
    regex = None # The matching is done based on open/close tag

    def __init__(self, open_tag: RangeOpenTag, close_tag: RangeCloseTag):
        super().__init__(open_tag, close_tag)

    def execute(self, content: str) -> Union[str, None]:
        # Manipulate lines between the tags
  1. Add the new tag(s) to the default_tags in the tokenizer,
default_tags: Set[Type[SingleTag]] = {
    IgnoreFileTag,
    RemoveOpenTag,
    ...,
    TestTag,
    TestOpenTag
}

⚠️ Only the SingleTag(s) need to be added, not the RangeTag