WIKI.md

Welcome to the CogAlg wiki!

Much of the coding was done by Khanh Nguyen and Kok Wei Chee, according to the principles introduced in README.

Main principle is input cross-comparison, then clustering by resulting match. This is done on all levels of composition from hierarchical clustering. First level cross-comp (cross-correlation) is among pixels, computing derivatives of brightness in a sliding kernel. Next step is segmentation: pixels + pixel-level derivatives ("derts") are clustered into positive and negative patterns: contiguous spans of above or below- average match.

To preserve positional info for evaluation, such algorithm must be specific to external (Cartesian) dimensionality of the input. Thus, we have three self-contained dimensionality-specific folders with separate workflow, explained below. Beyond 1D, derivatives per dimension are combined into gradient. Cross-comparison here is very similar to basic edge-detection operator, resulting patterns are blobs of same-sign deviation of kernel-level match or gradient.

Design and exploration of this algorithm is done in the following sequence:

line_1D_alg:

• line_Ps (old stand-alone version: line_patterns) is 1st-level core algorithm for 1D input: horizontal rows of pixels within an image. It forms patterns: line segments with same-sign deviation of difference between pixels, as well as sub-patterns by divisive hierarchical clustering. Sub-patterns are formed by recursive incremental range or incremental derivation cross-comp within selected patterns.
• line_PPs is a 2nd-level 1D algorithm, mostly done. It cross-compares each parameter of 1st-level patterns and forms Pps: param patterns, which in the aggregate represent patterns of patterns: PPs. It contains extended versions of 1st-level functions, as well as some new ones to handle intra-pattern hierarchy.
• line_recursive is for level-recursion from 3rd level and higher. It forms incrementally higher levels of pattern composition and syntactic complexity, also by cross-comparison and agglomerative clustering. This should be final a module of 1D algorithm, indefinitely scalable in the complexity of discoverable patterns, still tentative.

1D algorithm is a prototype, exclusively 1D patterns are not terribly informative / predictive in our 4D space-time. But it's best to develop basic principles and operations, which can then be converted to work in higher dimensions.

frame_2D_alg:

1st level: Chart.

• frame_blobs defines initial blobs: contiguous areas of same-sign deviation of gradient per pixel. It's a basic cross-correlation and connectivity clustering within an image, but resulting blobs also contain comparison-derived parameters for future comparison between blobs (comp_blob). This code is functional.

• frame_blobs_par is POC of parallel-processing version. It is currently not much faster, but critical for potential scalability.

• intra_blob: recursive calls to intra_comp: cross-comparison at extended range or higher derivation within selected blobs and then sub_blobs. Each call converts input blob into root blob and forms respective sub_blobs: contiguous areas of same-sign deviation of a new gradient.

  Diagram,

  • intra_comp cross-compares pixels over extended range, or cross-compares angle of gradient, forming corresponding type of new gradient.

  • draw_intra_comp computes gradients of the first three layers of forks over the whole frame, for visualization only (old version).

  • comp_slice will be terminal fork of intra_blob, selective for smooth elongated high-gradient blobs: likely edges or contours. It forms edge-orthogonal Ps: horizontal blob slices, then cross-compares vertically consecutive Ps. This is a 2D -> 1D dimensionality reduction, converting edges into vector representations. It is similar to the second level of 1D alg, which cross-compares horizontally discontinuous Ps: line_PPs_draft.

2nd level is frame_bblobs: cross-comp of blobs formed by frame_blobs, forming higher-composition bblobs, currently a draft. Final module will be frame_recursive: compositionally recursive agglomerative clustering, a 2D analog of line_recursive in 1D. It will cross-compare top layer of each input blob, of whatever composition order, with conditional specification by cross-comp of deeper layers if input value + match > average. Clustering the blobs (by match only?) forms incrementally higher-composition super-blobs: bblobs, bbblobs, etc.

video_3D_alg:

• video_draft() is obsolete 3D extension of pixel-level cross-correlation, as in frame_blobs. Eventually, it will extend all of 2D alg with time dimension. This version will be made fully recursive and hopefully effective in real world.

Higher levels of each D-cycle algorithm will do discontinuous search among dimensionally-complete patterns: blobs or graphs. The most complex part there will be cross-comparison, because it has to operate on multi-level, multi-variate input patterns, with multiple arithmetic powers (such as comparison by division). And each comparison will have variable and generally expanding range, resulting in looser proximity clustering, vs. simple connectivity clustering on the 1st level. Discontinuity in clustering results from that in cross-comp.

Complete hierarchical algorithm will have two-level code:

• 1st level algorithm: contiguous cross-comparison and clustering over full-dimensional cycle (up to 4D), plus feedback to adjust most and least significant bits of the input.
• Recurrent increment in complexity, extending current-level alg to next-level alg. This increment will account for increasing internal complexity of input patterns on higher levels, unfolding them for cross-comparison, then re-combining the results for evaluation and feedback of average deviation of each derivative.

We will then add colors, maybe audio and text. Initial testing could be recognition of labeled images, but 2D is also a poor representation of our 4D world, we will probably start directly with video or stereo video.

Suggestions and collaboration are most welcome, see CONTRIBUTING.